bug#62713: 29.0.60; DocView shows wrong conversion indicator when configured to use svg images

bug#62713: 29.0.60; DocView shows wrong conversion indicator when configured to use svg images

[Gregor Zattler]

Package: emacs
Version: 29.0.60
Severity: minor

Dear Emacs developers,

1. start emacs -Q,
2. evaluate `(setq doc-view-mupdf-use-svg t)' in *scratch* buffer, then
3. open a (preferably large) pdf file:

While the pdf file is converted to images in the
background, a modeline indicator is shown:

DocView:pdf->png

which is wrong in this case since the conversion is
configured to produce svg images.

The indicator should look like

DocView: pdf->svg

or

DocView:pdf->img

or

DocView:pdf conversion

or something like that.

Admittedly a very minor issue.
Thanks, Gregor



In GNU Emacs 29.0.60 (build 9, x86_64-pc-linux-gnu,
 cairo version 1.16.0) of 2023-03-18 built on no
Repository revision: 11592bcfda6cf85d797d333072453c98994790e1
Repository branch: HEAD
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Debian GNU/Linux 11 (bullseye)

Configured using:
 'configure --with-sound=yes --without-gconf
 --with-mailutils --build x86_64-linux-gnu
 --infodir=/usr/share/info --with-json
 --with-file-notification=yes --with-cairo --with-x=yes
 --with-x-toolkit=no --without-toolkit-scroll-bars
 --enable-checking=yes,glyphs
 --enable-check-lisp-object-type
 --with-native-compilation 'CFLAGS=-g3 -Os
 -ffile-prefix-map=/home/grfz/src/emacs-emacs-29--11592bcfda6cf85d797d333072453c98994790e1--2023-03-18T17-21+01-00=. -fstack-protector-strong
 -Wformat -Werror=format-security ''

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM
GSETTINGS HARFBUZZ JPEG JSON LCMS2 LIBOTF LIBSELINUX
LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP NOTIFY
INOTIFY OLDXMENU PDUMPER PNG RSVG SECCOMP SOUND SQLITE3
THREADS TIFF X11 XDBE XIM XINPUT2 XPM ZLIB

bug#62713: 29.0.60; DocView shows wrong conversion indicator when configured to use svg images

[Stefan Kangas]

Gregor Zattler <grfz@gmx.de> writes:

> Package: emacs
> Version: 29.0.60
> Severity: minor
>
> Dear Emacs developers,
>
> 1. start emacs -Q,
> 2. evaluate `(setq doc-view-mupdf-use-svg t)' in *scratch* buffer, then
> 3. open a (preferably large) pdf file:
>
> While the pdf file is converted to images in the
> background, a modeline indicator is shown:
>
> DocView:pdf->png
>
> which is wrong in this case since the conversion is
> configured to produce svg images.
>
> The indicator should look like
>
> DocView: pdf->svg
>
> or
>
> DocView:pdf->img
>
> or
>
> DocView:pdf conversion
>
> or something like that.
>
> Admittedly a very minor issue.
> Thanks, Gregor

Visuwesh, what do you think?

> In GNU Emacs 29.0.60 (build 9, x86_64-pc-linux-gnu,
>  cairo version 1.16.0) of 2023-03-18 built on no
> Repository revision: 11592bcfda6cf85d797d333072453c98994790e1
> Repository branch: HEAD
> Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
> System Description: Debian GNU/Linux 11 (bullseye)
>
> Configured using:
>  'configure --with-sound=yes --without-gconf
>  --with-mailutils --build x86_64-linux-gnu
>  --infodir=/usr/share/info --with-json
>  --with-file-notification=yes --with-cairo --with-x=yes
>  --with-x-toolkit=no --without-toolkit-scroll-bars
>  --enable-checking=yes,glyphs
>  --enable-check-lisp-object-type
>  --with-native-compilation 'CFLAGS=-g3 -Os
>  -ffile-prefix-map=/home/grfz/src/emacs-emacs-29--11592bcfda6cf85d797d333072453c98994790e1--2023-03-18T17-21+01-00=. -fstack-protector-strong
>  -Wformat -Werror=format-security ''
>
> Configured features:
> ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM
> GSETTINGS HARFBUZZ JPEG JSON LCMS2 LIBOTF LIBSELINUX
> LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP NOTIFY
> INOTIFY OLDXMENU PDUMPER PNG RSVG SECCOMP SOUND SQLITE3
> THREADS TIFF X11 XDBE XIM XINPUT2 XPM ZLIB

bug#62713: 29.0.60; DocView shows wrong conversion indicator when configured to use svg images

[Visuwesh]

Good day, Stefan!

[ஞாயிறு செப்டம்பர் 24, 2023] Stefan Kangas wrote:

>> The indicator should look like
>>
>> DocView: pdf->svg
>>
>> or
>>
>> DocView:pdf->img
>>
>> or
>>
>> DocView:pdf conversion
>>
>> or something like that.
>>
>> Admittedly a very minor issue.
>> Thanks, Gregor
>
> Visuwesh, what do you think?

It is better solved by writing a separate pdf->svg converter function.
Right now, the svg conversion is tacked on top of pdf->png converter
function used for MuPDF.  This was done because I thought only MuPDF
supported pdf->svg conversion but later I was informed that poppler has
utilities to do too.

Such a generic approach was discussed in [1] with the other Stefan but
it fell off my radar since I rarely use docview and hence lost my
motivation.  I will try to whip something up in the coming week.

1. https://yhetil.org/emacs-bugs/87zganwvgp.fsf@gmail.com/

bug#62713: 29.0.60; DocView shows wrong conversion indicator when configured to use svg images

[Stefan Kangas]

Visuwesh <visuweshm@gmail.com> writes:

> It is better solved by writing a separate pdf->svg converter function.
> Right now, the svg conversion is tacked on top of pdf->png converter
> function used for MuPDF.  This was done because I thought only MuPDF
> supported pdf->svg conversion but later I was informed that poppler has
> utilities to do too.
>
> Such a generic approach was discussed in [1] with the other Stefan but
> it fell off my radar since I rarely use docview and hence lost my
> motivation.  I will try to whip something up in the coming week.

Sounds good!  Thanks for working on this.

bug#62713: 29.0.60; DocView shows wrong conversion indicator when configured to use svg images

[Visuwesh]

[-- Attachment #1: Type: text/plain, Size: 1879 bytes --]

[திங்கள் செப்டம்பர் 25, 2023] Visuwesh wrote:

> It is better solved by writing a separate pdf->svg converter function.
> Right now, the svg conversion is tacked on top of pdf->png converter
> function used for MuPDF.  This was done because I thought only MuPDF
> supported pdf->svg conversion but later I was informed that poppler has
> utilities to do too.

And looks like the poppler utility pdftocairo does not work the doc-view
expects it to when generating svg images.  AFAICT, there is no way to
make pdftocairo to generate a svg image per page like other tools do.  I
cannot loop either as there is also no way to know the number of pages
beforehand.  The png function also has hacks since pdftocairo is not
cooperative, and changing doc-view to not use
doc-view--image-file-pattern does not seem to be easy.  I attach the
patch named "doc-view-general-svg-wip.patch" which has my WIP and it
works just fine for MuPDF.

This particular problem can be easily solved by the following simple
patch instead:

diff --git a/lisp/doc-view.el b/lisp/doc-view.el
index b856b09c8b7..c0d27bdc68a 100644
--- a/lisp/doc-view.el
+++ b/lisp/doc-view.el
@@ -1283,7 +1283,8 @@ doc-view-pdf->png-converter-mupdf
                                     (expand-file-name
                                      doc-view-epub-user-stylesheet)))))))
     (doc-view-start-process
-     "pdf->png" doc-view-pdfdraw-program
+     (concat "pdf->" (symbol-name doc-view--image-type))
+     doc-view-pdfdraw-program
      `(,@(doc-view-pdfdraw-program-subcommand)
        ,@options
        ,pdf

But if you say we don't bother about pdftocairo support and implement
the generic support system, I can prepare a patch without the pdftocairo
stuff, and add some documentation to aid the users to write custom
converter functions.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: doc-view-general-svg-wip.patch --]
[-- Type: text/x-diff, Size: 8409 bytes --]

diff --git a/lisp/doc-view.el b/lisp/doc-view.el
index b856b09c8b7..03b01de5033 100644
--- a/lisp/doc-view.el
+++ b/lisp/doc-view.el
@@ -193,28 +193,63 @@ doc-view-pdfdraw-program
   :type 'file
   :version "24.4")

+(defcustom doc-view-poppler-program "pdftocairo"
+  "Name of pdfcairo program from Poppler."
+  :type 'file
+  :version "30.1")
+
 (defcustom doc-view-pdftotext-program-args '("-raw")
   "Parameters to give to the pdftotext command."
   :version "27.1"
   :type '(repeat string))

 (defcustom doc-view-pdf->png-converter-function
-  (if (executable-find doc-view-pdfdraw-program)
-      #'doc-view-pdf->png-converter-mupdf
-    #'doc-view-pdf->png-converter-ghostscript)
+  (cond
+   ((executable-find doc-view-poppler-program)
+    #'doc-view-pdf->png-converter-poppler)
+   ((executable-find doc-view-pdfdraw-program)
+    #'doc-view-pdf->png-converter-mupdf)
+   (t #'doc-view-pdf->png-converter-ghostscript))
   "Function to call to convert a PDF file into a PNG file."
   :type '(radio
           (function-item doc-view-pdf->png-converter-ghostscript
                          :doc "Use ghostscript")
           (function-item doc-view-pdf->png-converter-mupdf
                          :doc "Use mupdf")
+          (function-item doc-view-pdf->png-converter-poppler
+                         :doc "Use pdfcairo from poppler")
           function)
   :version "24.4")

+(defcustom doc-view-pdf->svg-converter-function
+  (cond
+   ((executable-find doc-view-poppler-program)
+    #'doc-view-pdf->svg-converter-poppler)
+   ((executable-find doc-view-pdfdraw-program)
+    #'doc-view-pdf->svg-converter-mupdf)
+   (t #'doc-view-pdf->svg-converter-mupdf))
+  "Function to call to convert a PDF file into a SVG file."
+  :type '(radio
+          (function-item doc-view-pdf->svg-converter-mupdf
+                         :doc "Use mupdf")
+          (function-item doc-view-pdf->svg-converter-poppler
+                         :doc "Use pdfcairo from poppler")
+          function)
+  :version "30.1")
+
+(defcustom doc-view-use-svg (image-type-available-p 'svg)
+  "Whether to use svg images for files whenever possible."
+  :type 'boolean
+  :version "30.1")
+
 (defcustom doc-view-mupdf-use-svg (image-type-available-p 'svg)
   "Whether to use svg images for PDF files."
   :type 'boolean
-  :version "30.1")
+  :version "30.1"
+  :set (lambda (sym val)
+         (set-default-toplevel-value sym val)
+         (setq doc-view-use-svg val)))
+(make-obsolete 'doc-view-mupdf-use-svg 'doc-view-use-svg "30.1")

 (defcustom doc-view-imenu-enabled (and (executable-find "mutool") t)
   "Whether to generate an imenu outline when \"mutool\" is available."
@@ -241,7 +276,7 @@ doc-view-imenu-flatten
 (defface doc-view-svg-face '((t :inherit default))
   "Face used for SVG images.  Only background and foreground colors
 are used.
-See `doc-view-mupdf-use-svg'."
+See `doc-view-use-svg'."
   :version "30.1")

 (make-obsolete 'doc-view-svg-background 'doc-view-svg-face "30.1")
@@ -1266,10 +1301,10 @@ doc-view-pdf-password-protected-pdfdraw-p
     (goto-char (point-min))
     (search-forward "error: cannot authenticate password" nil t)))

-(defun doc-view-pdf->png-converter-mupdf (pdf png page callback)
+(defun doc-view-pdf->img-converter-mupdf (pdf type img page callback)
   (let* ((pdf-passwd (if (doc-view-pdf-password-protected-pdfdraw-p pdf)
                          (read-passwd "Enter password for PDF file: ")))
-         (options `(,(concat "-o" png)
+         (options `(,(concat "-o" img)
                     ,(format "-r%d" (round doc-view-resolution))
                     ,@(if pdf-passwd `("-p" ,pdf-passwd)))))
     (when (eq doc-view-doc-type 'epub)
@@ -1283,13 +1318,74 @@ doc-view-pdf->png-converter-mupdf
                                     (expand-file-name
                                      doc-view-epub-user-stylesheet)))))))
     (doc-view-start-process
-     "pdf->png" doc-view-pdfdraw-program
+     (concat "pdf->" type) doc-view-pdfdraw-program
      `(,@(doc-view-pdfdraw-program-subcommand)
        ,@options
        ,pdf
        ,@(if page `(,(format "%d" page))))
      callback)))

+(defun doc-view-pdf->png-converter-mupdf (pdf img page callback)
+  (doc-view-pdf->img-converter-mupdf pdf "png" img page callback))
+
+(defun doc-view-pdf->svg-converter-mupdf (pdf img page callback)
+  (doc-view-pdf->img-converter-mupdf pdf "svg" img page callback))
+
+(defun doc-view-pdf-password-protected-poppler-p (pdf)
+  "Return non-nil if a PDF file is password-protected."
+  (with-temp-buffer
+    (let* ((tmp (make-temp-name "emacs-doc-view-test"))
+           (abs (expand-file-name (concat tmp ".png") temporary-file-directory)))
+      (call-process doc-view-poppler-program nil (current-buffer) nil
+                    "-png" "-f" "1" "-singlefile" pdf abs)
+      (when (file-regular-p abs)
+        (delete-file abs))
+      (goto-char (point-min))
+      (search-forward "Incorrect password" nil t))))
+
+(defun doc-view-pdf->png-converter-poppler (pdf imgfile page callback)
+  (let ((passwd (when (doc-view-pdf-password-protected-poppler-p pdf)
+                  (read-passwd "Enter password for PDF file: "))))
+    ;; HACK: pdftocairo doesn't accept format arguments in the image
+    ;; filename argument for PNG, JPEG, TIFF, and doc-view relies on a
+    ;; valid `doc-view--image-file-pattern' that can be passed to
+    ;; `format' to change pages in `doc-view-goto-page'.  But
+    ;; thankfully, when PAGE is nil it means to fetch all pages.
+    (when (and (null page) (string-match-p "%d" imgfile))
+      (setq imgfile (string-remove-suffix
+                     (concat "-%d." (symbol-name doc-view--image-type))
+                     imgfile)))
+    (doc-view-start-process
+     "pdf->png"
+     doc-view-poppler-program
+     `("-png" "-r" ,(format "%d" (round doc-view-resolution))
+              ,@(when passwd `("-opw" ,passwd))
+              ,pdf
+              ,@(if page         ; Single page.
+                    (list "-f" (format "%d" page) "-singlefile"
+                          (file-name-sans-extension imgfile))
+                  (list imgfile)))
+     callback)))
+
+(defun doc-view-pdf->svg-converter-poppler (pdf imgfile page callback)
+  (let ((passwd (when (doc-view-pdf-password-protected-poppler-p pdf)
+                  (read-passwd "Enter password for PDF file: "))))
+    ;; For vector formats such as svg, output-file name is handled
+    ;; differently.  *Sigh*
+    ;; There seems to be no way to generate separate image per-page so
+    ;; we give up?
+    (unless (null page)
+      (doc-view-start-process
+       "pdf->svg"
+       doc-view-poppler-program
+       `("-svg" "-r" ,(format "%d" (round doc-view-resolution))
+         ,@(when passwd `("-opw" ,passwd))
+         ,pdf
+         ;; -singlefile is not supported for svg...
+         "-f" ,(format "%d" page) "-l" ,(format "%d" page)
+         ,imgfile)
+       callback))))
+
 (defun doc-view-odf->pdf-converter-unoconv (odf callback)
   "Convert ODF to PDF asynchronously and call CALLBACK when finished.
 The converted PDF is put into the current cache directory, and it
@@ -1329,7 +1425,9 @@ doc-view-pdf/ps->png
   (funcall
    (pcase doc-view-doc-type
      ((or 'pdf 'odf 'epub 'cbz 'fb2 'xps 'oxps)
-      doc-view-pdf->png-converter-function)
+      (if doc-view-use-svg
+          doc-view-pdf->svg-converter-function
+        doc-view-pdf->png-converter-function))
      ('djvu #'doc-view-djvu->tiff-converter-ddjvu)
      (_ #'doc-view-ps->png-converter-ghostscript))
    pdf-ps png nil
@@ -2148,10 +2246,8 @@ doc-view-set-up-single-converter
   (pcase-let ((`(,conv-function ,type ,extension)
                (pcase doc-view-doc-type
                  ('djvu (list #'doc-view-djvu->tiff-converter-ddjvu 'tiff "tif"))
-                 (_ (if (and (eq doc-view-pdf->png-converter-function
-                                 #'doc-view-pdf->png-converter-mupdf)
-                             doc-view-mupdf-use-svg)
-                        (list doc-view-pdf->png-converter-function 'svg "svg")
+                 (_ (if doc-view-use-svg
+                        (list doc-view-pdf->svg-converter-function 'svg "svg")
                       (list doc-view-pdf->png-converter-function 'png "png"))))))
     (setq-local doc-view-single-page-converter-function conv-function)
     (setq-local doc-view--image-type type)

bug#62713: 29.0.60; DocView shows wrong conversion indicator when configured to use svg images

[Stefan Kangas]

Visuwesh <visuweshm@gmail.com> writes:

> [திங்கள் செப்டம்பர் 25, 2023] Visuwesh wrote:
>
>> It is better solved by writing a separate pdf->svg converter function.
>> Right now, the svg conversion is tacked on top of pdf->png converter
>> function used for MuPDF.  This was done because I thought only MuPDF
>> supported pdf->svg conversion but later I was informed that poppler has
>> utilities to do too.
>
> And looks like the poppler utility pdftocairo does not work the doc-view
> expects it to when generating svg images.  AFAICT, there is no way to
> make pdftocairo to generate a svg image per page like other tools do.  I
> cannot loop either as there is also no way to know the number of pages
> beforehand.  The png function also has hacks since pdftocairo is not
> cooperative, and changing doc-view to not use
> doc-view--image-file-pattern does not seem to be easy.  I attach the
> patch named "doc-view-general-svg-wip.patch" which has my WIP and it
> works just fine for MuPDF.
>
> This particular problem can be easily solved by the following simple
> patch instead:

Tassilo, could you perhaps take a look at this patch?

bug#62713: 29.0.60; DocView shows wrong conversion indicator when configured to use svg images

[Tassilo Horn]

Stefan Kangas <stefankangas@gmail.com> writes:

>> This particular problem can be easily solved by the following simple
>> patch instead:
>
> Tassilo, could you perhaps take a look at this patch?

Yes, the simple patch looks good and fixes the bug.  Feel free to commit
it.

I've skimmed at the WIP patch, too.  It also looks good but has not much
to do with the bug report but adds another pdf->svg converter.
Visuwesh, I noticed that the docstrings speak of pdfcairo while the
program actually is pdftocairo.

Bye,
Tassilo

bug#62713: 29.0.60; DocView shows wrong conversion indicator when configured to use svg images

[Stefan Kangas]

Tassilo Horn <tsdh@gnu.org> writes:

> Stefan Kangas <stefankangas@gmail.com> writes:
>
>>> This particular problem can be easily solved by the following simple
>>> patch instead:
>>
>> Tassilo, could you perhaps take a look at this patch?
>
> Yes, the simple patch looks good and fixes the bug.  Feel free to commit
> it.

Great, thanks.

Visuwesh, could you please send the patch as an attachment instead?  We
prefer that patches are created with a command like `git format-patch
-1'.

Please also include:

- The bug number of this bug in the commit message, like so: Bug#62713

- A ChangeLog entry, as described in the file CONTRIBUTE.

That would make it easier for us to review and install this patch.

Thanks in advance.

bug#62713: 29.0.60; DocView shows wrong conversion indicator when configured to use svg images

[Visuwesh]

[-- Attachment #1: Type: text/plain, Size: 644 bytes --]

[வெள்ளி செப்டம்பர் 29, 2023] Stefan Kangas wrote:

> Tassilo Horn <tsdh@gnu.org> writes:
>
>> Stefan Kangas <stefankangas@gmail.com> writes:
>>
>>>> This particular problem can be easily solved by the following simple
>>>> patch instead:
>>>
>>> Tassilo, could you perhaps take a look at this patch?
>>
>> Yes, the simple patch looks good and fixes the bug.  Feel free to commit
>> it.
>
> Great, thanks.
>
> Visuwesh, could you please send the patch as an attachment instead?  We
> prefer that patches are created with a command like `git format-patch
> -1'.

Please find attached patch.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-doc-view-conversion-indicator-for-svg-images.patch --]
[-- Type: text/x-diff, Size: 994 bytes --]

From a73ab9af07f5867da3b6ff97c73d35230ea21c1e Mon Sep 17 00:00:00 2001
From: Visuwesh <visuweshm@gmail.com>
Date: Fri, 29 Sep 2023 20:23:56 +0530
Subject: [PATCH] Fix doc-view conversion indicator for svg images

* lisp/doc-view.el (doc-view-pdf->png-converter-mupdf): Show the
corrector pdf->svg indicator when producing svg images.  bug#62713
---
 lisp/doc-view.el | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lisp/doc-view.el b/lisp/doc-view.el
index b856b09c8b7..c0d27bdc68a 100644
--- a/lisp/doc-view.el
+++ b/lisp/doc-view.el
@@ -1283,7 +1283,8 @@ doc-view-pdf->png-converter-mupdf
                                     (expand-file-name
                                      doc-view-epub-user-stylesheet)))))))
     (doc-view-start-process
-     "pdf->png" doc-view-pdfdraw-program
+     (concat "pdf->" (symbol-name doc-view--image-type))
+     doc-view-pdfdraw-program
      `(,@(doc-view-pdfdraw-program-subcommand)
        ,@options
        ,pdf
-- 
2.40.1

bug#62713: 29.0.60; DocView shows wrong conversion indicator when configured to use svg images

[Stefan Kangas]

Version: 30.1

Visuwesh <visuweshm@gmail.com> writes:

> [வெள்ளி செப்டம்பர் 29, 2023] Stefan Kangas wrote:
>
>> Tassilo Horn <tsdh@gnu.org> writes:
>>
>>> Stefan Kangas <stefankangas@gmail.com> writes:
>>>
>>>>> This particular problem can be easily solved by the following simple
>>>>> patch instead:
>>>>
>>>> Tassilo, could you perhaps take a look at this patch?
>>>
>>> Yes, the simple patch looks good and fixes the bug.  Feel free to commit
>>> it.
>>
>> Great, thanks.
>>
>> Visuwesh, could you please send the patch as an attachment instead?  We
>> prefer that patches are created with a command like `git format-patch
>> -1'.
>
> Please find attached patch.

Thanks, installed on master [1: f0abc9bc0d5], and I'm consequently
closing this bug report.

[1: f0abc9bc0d5]: 2023-10-01 14:58:48 +0200
  Fix doc-view conversion indicator for svg images
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=f0abc9bc0d536e5b2834c04714d75d18fde431c3