[HELP] Packaging mupdf

[HELP] Packaging mupdf

[Pierre-Henry F.]

[-- Attachment #1: Type: text/plain, Size: 1267 bytes --]

Hello,

I'm trying to learn how to package code using Guix and packaging
mupdf doing do. I need some help to continue.

I've followed packaging tutorial for guix[1] and hopefully know
enough scheme (Racket in fact) to carry one.

The problem with the way mupdf is package today[2] is that it
does not include the patched version of freeglut that is
necessary for the copy-pasting functionality[3].

What does work is to follow the build instructions of mupdf[4]
which boils down to:

  git clone --recursive git://git.ghostscript.com/mupdf.git
  cd mupdf
  git submodule update --init
  sudo apt-get install mesa-common-dev libgl1-mesa-dev libglu1-mesa-dev xorg-dev libxcursor-dev libxrandr-dev libxinerama-dev
  make prefix=~/bin/mupdf install

So, I guess that the objective is to somehow make guix execute
the above script.

I've been wondering around in the doc of guix w/o much success.
Is there anyone that could help me out? I can document the thing
and post it on my blog or give it back here for anyone to do
anything with it, should it be of some use.

Many thanks,
PH

[1]: https://www.gnu.org/software/guix/blog/2018/a-packaging-tutorial-for-guix/
[2]: $ guix edit mupdf
[3]: https://bugs.archlinux.org/task/57227
[4]: https://mupdf.com/docs/building.html

[-- Attachment #2: Type: text/html, Size: 2040 bytes --]

Re: [HELP] Packaging mupdf

[Ricardo Wurmus]

Hi Pierre-Henry,

> The problem with the way mupdf is package today[2] is that it
> does not include the patched version of freeglut that is
> necessary for the copy-pasting functionality[3].
>
> What does work is to follow the build instructions of mupdf[4]
> which boils down to:
>
>   git clone --recursive git://git.ghostscript.com/mupdf.git
>   cd mupdf
>   git submodule update --init
>   sudo apt-get install mesa-common-dev libgl1-mesa-dev libglu1-mesa-dev xorg-dev libxcursor-dev libxrandr-dev libxinerama-dev
>   make prefix=~/bin/mupdf install
>
> So, I guess that the objective is to somehow make guix execute
> the above script.

No, this would not work and would not be desirable.

Our mupdf package is already built with freeglut.  Instead of bundling a
patched version of freeglut with your variant of mupdf it would better
to keep the packages separate.

Your first step would be to create a package variant of freeglut (use
“inherit” to avoid duplication) that includes the patch — or maybe it
would make sense to patch freeglut for all its users, I don’t know.

Once that is done you can refer to the new freegut variant in your mupdf
variant package.

Let me get back to the snippet you showed us:

>   git clone --recursive git://git.ghostscript.com/mupdf.git

We express this with the package’s “source” field.

    (source (origin
              (method git-fetch)
              (uri (git-reference
                    (url "git://git.ghostscript.com/mupdf.git")
                    (commit the-commit-you-need)))
              (file-name (git-file-name name version))
              (sha256
               (base32
                "…the result of guix hash -rx . in the checkout…"))))

Let’s not do the recursive clone here, because we don’t actually want
to include all of the submodules which bundle third party code.

>   cd mupdf

We can ignore this.  The “unpack” build phase takes care of this
already.

>   git submodule update --init

We don’t do that.  If we really wanted to fetch submodules we would
specify in the origin above that we want a recursive clone.

>   sudo apt-get install mesa-common-dev libgl1-mesa-dev
>   libglu1-mesa-dev xorg-dev libxcursor-dev libxrandr-dev
>   libxinerama-dev

Since we aren’t using Debian we don’t run this ;)
Instead, we express this through the “inputs” and “native-inputs”
fields.

>   make prefix=~/bin/mupdf install

The usual steps of the GNU build system (configure, make, make check,
make install) are implemented in the gnu-build-system, which we specify
as the value for the “build-system” field.

I encourage you to take a look at gnu/packages/pdf.scm, which contains a
definition for mupdf.  It uses the gnu-build-system and changes its
behaviour via the “arguments” field (e.g. to pass extra options to
“make”, to delete build phases that don’t make sense here, to disable
tests, etc).

Hope this helps!

--
Ricardo

Re: [HELP] Packaging mupdf

[Pierre-Henry F.]

Thank you for your reply!

I documented my wondering around below and here is the gist of it:

To update the ~freeglut~ package (found with: ~$ guix edit freeglut~) may mean to
make guix do (the equivalent of) this:

#+begin_src sh
$ git clone --recursive git://git.ghostscript.com/mupdf.git
$ cd mupdf
$ git submodule update --init
$ tar -zcf freeglut.tar.gz thirdparty/freeglut
#+end_src

so that the source field in the ~freeglut~ package:

#+begin_src scheme
(define-public freeglut
  (package
    (name "freeglut")
    (version "3.0.0")
    (source (origin
              (method url-fetch)
              (uri (string-append
                    "mirror://sourceforge/freeglut/freeglut/"
                    version "/freeglut-" version ".tar.gz"))
              (sha256 … )))
    …))
#+end_src

uses this ~freeglut.tar.gz~ instead of:

#+begin_src scheme
(source (origin
              (method url-fetch)
              (uri (string-append
                    "mirror://sourceforge/freeglut/freeglut/"
                    version "/freeglut-" version ".tar.gz"))
              (sha256 … )))
#+end_src

How to update the package to that it does this?

Thanks,
PH





*** mupdf

    Build and install ~mupdf~ with: ~$ guix package -i mupdf~

    ~mupdf~ doesn't work properly because of [[https://bugs.archlinux.org/task/57227][this bug]].

    To make ~mupdf~ work as expected, it should be built against a patched version of ~freeglut~.

    Let's check how ~mupdf~ is currently packaged: ~$ guix edit mupdf~.

    We notice this field that denotes a dependency to ~freeglut~:

    #+begin_src scheme
      (inputs
       `(…
         ("freeglut" ,freeglut)
         …))
    #+end_src

    According to the [[https://www.gnu.org/software/guix/manual/en/html_node/Defining-Packages.html#Defining-Packages][documentation]], the ~inputs~ field is a list of key-value pairs
    where keys are strings and values are packages. So, a ~freeglut~ package should
    be defined. Let's find it:

    ~$ guix package -s freeglut~

    gives two versions of ~freeglut~, boths of them with this field: ~location:
    gnu/packages/gl.scm:93:2~ wich tells us that ~#:use-module (gnu packages gl)~
    should be used where ~mupdf~ package is defined and it is.

    So, we need:
    - to update the ~mupdf~ package so that it uses the patched version of ~freeglut~
    - to update the ~freeglut~ package so that it uses patched version of ~freeglut~
    - find where to get the patched version of ~freeglut~ from.

**** Patched version of freeglut

     Googling things gives [[https://bugs.ghostscript.com/show_bug.cgi?id=699079][this link]] with this comment by Tor Andersson:

     #+begin_quote
     However, since you mention not being able to copy the selection at all, you should be
     aware that you MUST build with OUR copy of FreeGLUT. The standard system FreeGLUT
     does NOT have copy & paste support. We have added clipboard support to our fork of
     FreeGLUT. I have tried to get our additions adopted upstream, but the maintainers are
     slow to respond, so please use our version of freeglut when building mupdf.
     #+end_quote

     Tor Andersson seems to know what he is talking about since [[http://git.ghostscript.com/?p=mupdf.git;a=summary][he commits]] to the
     ~mupdf~ sources. The question becomes: how to fetch « OUR copy of FreeGLUT ».
     Let's fetch ~mupdf~ sources and check the ~freeglut~ history there:

     #+begin_src sh
    $ git clone --recursive git://git.ghostscript.com/mupdf.git
    $ cd mupdf
    $ git submodule update --init
    $ cd thirdparty/freeglut
    $ git log
     #+end_src

     We notice in the logs: ~Import freeglut 3.0.0 from tarball.~ and commits by Tor
     Andersson dealing with things useful for copy-pasting.

     So, we should modify the ~guix~ package definition of ~freeglut version 3.0.0~ so
     that it uses this patched version of the ~freeglut~ sources.

     Since the ~freeglut~ that interests us is a submodule of the ~mupdf~ sources,
     let's refresh our memory with [[https://git-scm.com/book/en/v2/Git-Tools-Submodules][git documentation]].  We learned that to get the
     sources, we should do something like this in the ~freeglut~ guix package
     definition:

     #+id: 25eff785-d077-430e-aa2e-bfeb0eb07285
     #+begin_src sh
    $ git clone --recursive git://git.ghostscript.com/mupdf.git
    $ cd mupdf
    $ git submodule update --init
     #+end_src

     Then use the sources from there.



‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Saturday, March 2, 2019 11:15 PM, Ricardo Wurmus <rekado@elephly.net> wrote:

>
>
> Hi Pierre-Henry,
>
> > The problem with the way mupdf is package today[2] is that it
> > does not include the patched version of freeglut that is
> > necessary for the copy-pasting functionality[3].
> > What does work is to follow the build instructions of mupdf[4]
> > which boils down to:
> > git clone --recursive git://git.ghostscript.com/mupdf.git
> > cd mupdf
> > git submodule update --init
> > sudo apt-get install mesa-common-dev libgl1-mesa-dev libglu1-mesa-dev xorg-dev libxcursor-dev libxrandr-dev libxinerama-dev
> > make prefix=~/bin/mupdf install
> > So, I guess that the objective is to somehow make guix execute
> > the above script.
>
> No, this would not work and would not be desirable.
>
> Our mupdf package is already built with freeglut. Instead of bundling a
> patched version of freeglut with your variant of mupdf it would better
> to keep the packages separate.
>
> Your first step would be to create a package variant of freeglut (use
> “inherit” to avoid duplication) that includes the patch — or maybe it
> would make sense to patch freeglut for all its users, I don’t know.
>
> Once that is done you can refer to the new freegut variant in your mupdf
> variant package.
>
> Let me get back to the snippet you showed us:
>
> > git clone --recursive git://git.ghostscript.com/mupdf.git
>
> We express this with the package’s “source” field.
>
> (source (origin
> (method git-fetch)
> (uri (git-reference
> (url "git://git.ghostscript.com/mupdf.git")
> (commit the-commit-you-need)))
> (file-name (git-file-name name version))
> (sha256
> (base32
> "…the result of guix hash -rx . in the checkout…"))))
>
> Let’s not do the recursive clone here, because we don’t actually want
> to include all of the submodules which bundle third party code.
>
> > cd mupdf
>
> We can ignore this. The “unpack” build phase takes care of this
> already.
>
> > git submodule update --init
>
> We don’t do that. If we really wanted to fetch submodules we would
> specify in the origin above that we want a recursive clone.
>
> > sudo apt-get install mesa-common-dev libgl1-mesa-dev
> > libglu1-mesa-dev xorg-dev libxcursor-dev libxrandr-dev
> > libxinerama-dev
>
> Since we aren’t using Debian we don’t run this ;)
> Instead, we express this through the “inputs” and “native-inputs”
> fields.
>
> > make prefix=~/bin/mupdf install
>
> The usual steps of the GNU build system (configure, make, make check,
> make install) are implemented in the gnu-build-system, which we specify
> as the value for the “build-system” field.
>
> I encourage you to take a look at gnu/packages/pdf.scm, which contains a
> definition for mupdf. It uses the gnu-build-system and changes its
> behaviour via the “arguments” field (e.g. to pass extra options to
> “make”, to delete build phases that don’t make sense here, to disable
> tests, etc).
>
> Hope this helps!
>
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Ricardo

Re: [HELP] Packaging mupdf

[Ricardo Wurmus]

Hi Pierre-Henry,

the mupdf release tarball includes the freeglut sources.  We can reuse
the mupdf package’s “source” field and add a build phase after 'unpack
to change directories.

Here’s how (untested):

--8<---------------cut here---------------start------------->8---
(define freeglut-for-mupdf
  (package
    (inherit freeglut)
    (source (origin
              (method url-fetch)
              (uri (string-append "https://mupdf.com/downloads/archive/"
                                  name "-" version "-source.tar.xz"))
              (sha256
               (base32
                "1psnz02w5p7wc1s1ma7vvjmkjfy641xvsh9ykaqzkk84dflnjgk0"))
              (modules '((guix build utils)))
              (snippet
               '(begin
                  (for-each
                   (lambda (dir)
                     (delete-file-recursively (string-append "thirdparty/" dir)))
                   '("curl" "freetype" "harfbuzz" "jbig2dec" "lcms2"
                     "libjpeg" "mujs" "openjpeg" "zlib"))
                  #t))))
    (arguments
     '(#:tests? #f ; there are none
       #:phases
       (modify-phases %standard-phases
         (add-after 'unpack 'chdir
           (lambda _ (chdir "thirdparty/freeglut") #t)))))))
--8<---------------cut here---------------end--------------->8---

Hope that helps!

--
Ricardo

Re: [HELP] Packaging mupdf

[Pierre-Henry F.]

Hi Ricardo,

With your help and a little of headbanging, it finally works !

Here are the relevant parts of the code.
I wrote my journey too, if it's of any help, I will gladly send it here.

Best,
PH


freeGLUT:
    (define-module (freeglut-for-mupdf)
      #:use-module (guix packages)
      #:use-module (guix download)
      #:use-module (gnu packages gl))

    (define-public freeglut-for-mupdf
      (package
       (inherit freeglut)
       (name "freeglut-for-mupdf")
       (version "1.14.0")
       (source (origin
                (method url-fetch)
                (uri (string-append "https://mupdf.com/downloads/archive/mupdf-" version "-source.tar.gz"))
                (sha256
                 (base32
                  "093p7lv6pgyymagn28n58fs0np928r0i5p2az9cc4gwccwx4hhy4"))
                (modules '((guix build utils)))
                (snippet
                 '(begin
                    (for-each
                     (lambda (dir)
                       (delete-file-recursively (string-append "thirdparty/" dir)))
                     '("curl" "freetype" "harfbuzz" "jbig2dec" "lcms2"
                       "libjpeg" "mujs" "openjpeg" "zlib"))
                    #t))))
       (arguments
        '(#:tests? #f ; there are none
                   #:phases
                   (modify-phases %standard-phases
                                  (add-after 'unpack 'chdir
                                             (lambda _ (chdir "thirdparty/freeglut") #t)))))))

mupdf:
    (define-module (my-mupdf)
      #:use-module (guix packages)
      #:use-module (guix download)
      #:use-module (freeglut-for-mupdf)
      #:use-module (gnu packages pdf)
      #:use-module (gnu packages curl)
      #:use-module (gnu packages fontutils)
      #:use-module (gnu packages gtk)
      #:use-module (gnu packages image)
      #:use-module (gnu packages xorg)
      #:use-module (gnu packages javascript)
      #:use-module (gnu packages tls)
      #:use-module (gnu packages compression))

    (define-public my-mupdf
      (package
       (inherit mupdf)
       (name "my-mupdf")
       (inputs
        `(("curl" ,curl)
          ("freeglut" ,freeglut-for-mupdf)
          ("freetype" ,freetype)
          ("harfbuzz" ,harfbuzz)
          ("jbig2dec" ,jbig2dec)
          ("libjpeg" ,libjpeg)
          ("libx11" ,libx11)
          ("libxext" ,libxext)
          ("mujs" ,mujs)
          ("openjpeg" ,openjpeg)
          ("openssl" ,openssl)
          ("zlib" ,zlib)))))





Cordialement,
Pierre-Henry FRÖHRING
+33 6 34 48 17 57
contact@phfrohring.com
Skype: pierre.henry.frohring

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Sunday, March 3, 2019 3:30 PM, Ricardo Wurmus <rekado@elephly.net> wrote:

>
>
> Hi Pierre-Henry,
>
> the mupdf release tarball includes the freeglut sources. We can reuse
> the mupdf package’s “source” field and add a build phase after 'unpack
> to change directories.
>
> Here’s how (untested):
>
> --8<---------------cut here---------------start------------->8---
> (define freeglut-for-mupdf
> (package
> (inherit freeglut)
> (source (origin
> (method url-fetch)
> (uri (string-append "https://mupdf.com/downloads/archive/"
> name "-" version "-source.tar.xz"))
> (sha256
> (base32
> "1psnz02w5p7wc1s1ma7vvjmkjfy641xvsh9ykaqzkk84dflnjgk0"))
> (modules '((guix build utils)))
> (snippet
> '(begin
> (for-each
> (lambda (dir)
> (delete-file-recursively (string-append "thirdparty/" dir)))
> '("curl" "freetype" "harfbuzz" "jbig2dec" "lcms2"
> "libjpeg" "mujs" "openjpeg" "zlib"))
> #t))))
> (arguments
> '(#:tests? #f ; there are none
> #:phases
> (modify-phases %standard-phases
> (add-after 'unpack 'chdir
> (lambda _ (chdir "thirdparty/freeglut") #t)))))))
> --8<---------------cut here---------------end--------------->8---
>
> Hope that helps!
>
> -------------------
>
> Ricardo