DJVU files and ImageMagick

DJVU files and ImageMagick

[Stephen Berman]

Should the following change be made?

diff --git a/lisp/image.el b/lisp/image.el
index 3aa3b0aa24..ba75bdf034 100644
--- a/lisp/image.el
+++ b/lisp/image.el
@@ -902,7 +902,7 @@ imagemagick-register-types
       (setq imagemagick--file-regexp re))))
 
 (defcustom imagemagick-types-inhibit
-  '(C HTML HTM INFO M TXT PDF)
+  '(C HTML HTM INFO M TXT PDF DJVU)
   "List of ImageMagick types that should never be treated as images.
 This should be a list of symbols, each of which should be one of
 the ImageMagick types listed by `imagemagick-types'.  The listed

Here's my reason for asking:

If Emacs is built without ImageMagick support and the ddjvu program
(from DjVuLibre) is found, DJVU files are automatically (via
auto-mode-alist) displayed in doc-view-mode.  If Emacs is built with
ImageMagick support (version 6 or 7), image-mode is used to display DJVU
files, because DJVU is one of the images type imagemagick-register-types
assigns to image-mode and adds the corresponding entry to
auto-mode-alist before the entry using doc-view-mode-maybe.

I build Emacs from master with ImageMagick support, but if I try to
visit a DJVU file (e.g. a ca. 200 page book of about 1.5 MB),
immediately almost all my system's RAM is consumed, bringing Emacs to a
halt, so it doesn't display the file, and I can recover the memory only
by killing Emacs.  In contrast, when I open the same DJVU file from the
shell using the ImageMagick 'display' command, it also uses most of the
system memory for a number of seconds but then does display the file and
I can exit the program normally and recover the memory.  (This is with
ImageMagick 7; I also tried it on another system using ImageMagick 6 and
there, but using 'display' from the shell was the same as the Emacs
behavior: almost all memory was consumed and the process had to be
killed.)  Can anyone here visit a comparably sized DJVU file in Emacs
built with ImageMagick and have it displayed?  If so, maybe my system
just lacks sufficient memory (it has 4GB RAM + 1 GB swap) -- though that
doesn't explain the difference between 'display' and Emacs+ImageMagick
with ImageMagick 7.

I can prevent Emacs from using ImageMagick with DJVU by customizing
either imagemagick-types-inhibit or imagemagick-enabled-types, but if
this is a general issue, maybe the default value of one of these should
be changed to prevent using ImageMagick with DJVU files.  I see that by
default, PDF is in both lists, whereas DJVU is only in the latter (which
comes from imagemagick-types according to what ImageMagick itself claims
to support).  And indeed, when I try to open a PDF file from the shell
with 'display', the process aborts, so I infer that's why PDF was added
to imagemagick-types-inhibit, and hence, if the problem with DJVU I see
is general, the same thing should be done for this file type (i.e., the
above patch).  (I normally view DJVU files in Emacs using the
djvu package from ELPA, which like doc-view uses DjVuLibre programs, not
ImageMagick, so it doesn't have this issue; but if I use C-x C-f or
Dired to visit a DJVU file, I get the problem.)

If this is just a problem of too little memory, an alternative to
changing the default value of imagemagick-types-inhibit is to add a note
to etc/PROBLEMS suggesting to customize this user option.

Steve Berman

Re: DJVU files and ImageMagick

[Eli Zaretskii]

> From: Stephen Berman <stephen.berman@gmx.net>
> Date: Sat, 02 Feb 2019 23:12:12 +0100
> 
> Should the following change be made?
> 
> diff --git a/lisp/image.el b/lisp/image.el
> index 3aa3b0aa24..ba75bdf034 100644
> --- a/lisp/image.el
> +++ b/lisp/image.el
> @@ -902,7 +902,7 @@ imagemagick-register-types
>        (setq imagemagick--file-regexp re))))
>  
>  (defcustom imagemagick-types-inhibit
> -  '(C HTML HTM INFO M TXT PDF)
> +  '(C HTML HTM INFO M TXT PDF DJVU)
>    "List of ImageMagick types that should never be treated as images.
>  This should be a list of symbols, each of which should be one of
>  the ImageMagick types listed by `imagemagick-types'.  The listed

I think ImageMagick is a troubled package, so replacing it with better
alternatives is generally a Good Thing.  Therefore, I agree with this
change, and would encourage others to speak up their views on this.

Thanks.

Re: DJVU files and ImageMagick

[joakim]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stephen Berman <stephen.berman@gmx.net>
>> Date: Sat, 02 Feb 2019 23:12:12 +0100
>> 
>> Should the following change be made?
>> 
>> diff --git a/lisp/image.el b/lisp/image.el
>> index 3aa3b0aa24..ba75bdf034 100644
>> --- a/lisp/image.el
>> +++ b/lisp/image.el
>> @@ -902,7 +902,7 @@ imagemagick-register-types
>>        (setq imagemagick--file-regexp re))))
>>  
>>  (defcustom imagemagick-types-inhibit
>> -  '(C HTML HTM INFO M TXT PDF)
>> +  '(C HTML HTM INFO M TXT PDF DJVU)
>>    "List of ImageMagick types that should never be treated as images.
>>  This should be a list of symbols, each of which should be one of
>>  the ImageMagick types listed by `imagemagick-types'.  The listed
>
> I think ImageMagick is a troubled package, so replacing it with better
> alternatives is generally a Good Thing.  Therefore, I agree with this
> change, and would encourage others to speak up their views on this.

I also agree.

Longer version:

I use Imagemagick in Emacs to view single djvu files, not the djvu
bundles. I can still do this if I customize the inhibit list.

Even if the Imagemagick support went away altogether, I could mostly use
an image converter to do the same thing.


>
> Thanks.
>
-- 
Joakim Verona
joakim@verona.se

Re: DJVU files and ImageMagick

[Roland Winkler]

On Sat, Feb 02 2019, Stephen Berman wrote:
> I build Emacs from master with ImageMagick support, but if I try to
> visit a DJVU file (e.g. a ca. 200 page book of about 1.5 MB),
> immediately almost all my system's RAM is consumed, bringing Emacs to a
> halt, so it doesn't display the file, and I can recover the memory only
> by killing Emacs.

I am just curious: why are all pages converted immediately to emacs'
internal format upon loading a multipage document (be it a djvu document
or any other format)?  This sounds like a misfeature to me.  I am not
surprised this quickly consums all available memory, say, if you wanted
to work with multiple djvu documents simultaneously.

The djvu package from GNU Elpa uses the command line tool ddjvu from
djvulibre to convert only the one page that is currently displayed.
This approach is very fast in my experience, and it is obviously by far
less demanding when working with many "many-pages" documents.

> (I normally view DJVU files in Emacs using the djvu package from ELPA

Really?  Hooray, someone else is using this package, too.  I thought I
was the very only one.

Roland

Re: DJVU files and ImageMagick

[Colin Baxter]

>>>>> Roland Winkler <winkler@gnu.org> writes:

    > On Sat, Feb 02 2019, Stephen Berman wrote:
    >> I build Emacs from master with ImageMagick support, but if I try
    >> to visit a DJVU file (e.g. a ca. 200 page book of about 1.5 MB),
    >> immediately almost all my system's RAM is consumed, bringing
    >> Emacs to a halt, so it doesn't display the file, and I can
    >> recover the memory only by killing Emacs.

    > I am just curious: why are all pages converted immediately to
    > emacs' internal format upon loading a multipage document (be it a
    > djvu document or any other format)?  This sounds like a misfeature
    > to me.  I am not surprised this quickly consums all available
    > memory, say, if you wanted to work with multiple djvu documents
    > simultaneously.

    > The djvu package from GNU Elpa uses the command line tool ddjvu
    > from djvulibre to convert only the one page that is currently
    > displayed.  This approach is very fast in my experience, and it is
    > obviously by far less demanding when working with many
    > "many-pages" documents.

    >> (I normally view DJVU files in Emacs using the djvu package from
    >> ELPA

    > Really?  Hooray, someone else is using this package, too.  I
    > thought I was the very only one.

Me too. I have built emacs with ImageMagick and can view 20Mb+
djvu files easily with the excellent djvu package.

Best wishes,

Colin Baxter
m43cap@yandex.com

Re: DJVU files and ImageMagick

[Stephen Berman]

On Sun, 03 Feb 2019 20:30:48 -0600 Roland Winkler <winkler@gnu.org> wrote:

> On Sat, Feb 02 2019, Stephen Berman wrote:
[...]
>> (I normally view DJVU files in Emacs using the djvu package from ELPA
>
> Really?  Hooray, someone else is using this package, too.  I thought I
> was the very only one.

I'm quite happy with this package, and the current version is even more
attractive compared to the previous one, especially the buttonized
outline.  There are a couple of features I miss; one is that I often
like to open DJVU files from Dired, so I added this to my init file:

(defun srb-djvu-mode ()
  (let ((file (buffer-file-name)))
    (kill-buffer (file-name-nondirectory file))
    (djvu-find-file file)))

(add-to-list 'auto-mode-alist '("\\.djvu\\'" . srb-djvu-mode))

Another nice thing would be integration with the Emacs bookmark facility
(I haven't tried using the djvu bookmark facility, but it looks like
it's tied to djvu outlines, so orthogonal to Emacs bookmarks).  I've
tried adapting the bookmark integration in Andreas Politz's pdf-tools
package, and it seems to works well enough for my purposes.  I also like
the way pdf-view-mode (and doc-view-mode) displays both the current and
total number of pages, so I adapted djvu-read-mode to that as well.
Here's the code, in case you or anyone else is interested:


(defun djvu-bookmark-make-record ()
  (cons (buffer-name)
       (append (bookmark-make-record-default nil t 1)
	       `(,(cons 'page (djvu-ref page))
		 ,(cons 'size (djvu-ref pagesize))
		 (handler . djvu-view-bookmark-jump-handler)))))

(defun djvu-view-bookmark-jump-handler (bmk)
  (let ((page (bookmark-prop-get bmk 'page))
        ;; (slice (bookmark-prop-get bmk 'slice))
        (size (bookmark-prop-get bmk 'size))
        ;; (origin (bookmark-prop-get bmk 'origin))
        (file (bookmark-prop-get bmk 'filename))
        (show-fn-sym (make-symbol "djvu-bookmark-after-jump-hook")))
    (fset show-fn-sym
          (lambda ()
            (remove-hook 'bookmark-after-jump-hook show-fn-sym)
            (unless (derived-mode-p 'djvu-read-mode)
              (djvu-read-mode))
            (with-selected-window
                (or (get-buffer-window (current-buffer) 0)
                    (selected-window))
              (when size
                (setq-local djvu-doc-pagesize size))
              ;; (when slice
              ;;   (apply 'pdf-view-set-slice slice))
              (when (numberp page)
                (djvu-goto-page page))
              ;; (when origin
              ;;   (let ((size (pdf-view-image-size t)))
              ;;     (image-set-window-hscroll
              ;;      (round (/ (* (car origin) (car size))
              ;;                (frame-char-width))))
              ;;     (image-set-window-vscroll
              ;;      (round (/ (* (cdr origin) (cdr size))
              ;;                (frame-char-height))))))
    	      )))
    (add-hook 'bookmark-after-jump-hook show-fn-sym)
    (set-buffer (or (find-buffer-visiting file)
                    (find-file-noselect file)))))

(define-derived-mode djvu-read-mode special-mode "Djview"
  "Mode for reading Djvu files."
  (setq djvu-buffer 'read
        buffer-undo-list t
        ;; mode-line-buffer-identification
        ;; (list 24 '(:eval (format "%s  p%d" (buffer-name) (djvu-ref page))))
	)
  (setq-local mode-line-position
              '(" P" (:eval (number-to-string (djvu-ref page)))
                ;; Avoid errors during redisplay.
                "/" (:eval (or (ignore-errors
                                 (number-to-string (djvu-ref pagemax)))
                               "???"))))
  (setq-local revert-buffer-function 'djvu-revert-buffer)
  (setq-local bookmark-make-record-function 'djvu-bookmark-make-record))


Thanks for the djvu package!

Steve Berman

Re: DJVU files and ImageMagick

[Roland Winkler]

On Mon, Feb 04 2019, Stephen Berman wrote:
> There are a couple of features I miss; one is that I often
> like to open DJVU files from Dired

Stefan suggested to me how to modify the code such that a djvu-mode can
be added to auto-mode-alist.  I have not yet found the time to look at
this more carefully.  But it should solve this problem.  In the
meanwhile you could advice find-file-noselect as described in djvu.el,
which should work in most cases, too.

> Another nice thing would be integration with the Emacs bookmark
> facility (I haven't tried using the djvu bookmark facility, but it
> looks like it's tied to djvu outlines, so orthogonal to Emacs
> bookmarks).

Djvu bookmarks are for efficiently moving within a djvu document,
similar to how one can define bookmarks within a pdf document.  I have
never used Emacs bookmarks, but it seems their main idea is different,
moving around across different files.

> I also like the way pdf-view-mode (and doc-view-mode) displays both
> the current and total number of pages

Yes, that makes sense; it will be part of the next update in GNU Elpa
(implemented slightly different from your code).

Roland

RE: DJVU files and ImageMagick

[Drew Adams]

> > Another nice thing would be integration with the Emacs bookmark
> > facility (I haven't tried using the djvu bookmark facility, but it
> > looks like it's tied to djvu outlines, so orthogonal to Emacs
> > bookmarks).
> 
> Djvu bookmarks are for efficiently moving within a djvu document,
> similar to how one can define bookmarks within a pdf document.  I have
> never used Emacs bookmarks, but it seems their main idea is different,
> moving around across different files.

I haven't been following this thread, and I know
nothing about djvu.

But the main idea of Emacs bookmarks is not about
moving around across different files.  The main idea
is just to record and return to a location.  Anywhere.
And a common use case is moving among locations in the
same buffer (e.g. same file).

(No idea whether Emacs bookmarks are relevant to djvu.)

Re: DJVU files and ImageMagick

[Stephen Berman]

On Tue, 05 Feb 2019 21:53:16 -0600 Roland Winkler <winkler@gnu.org> wrote:

> On Mon, Feb 04 2019, Stephen Berman wrote:
>> There are a couple of features I miss; one is that I often
>> like to open DJVU files from Dired
>
> Stefan suggested to me how to modify the code such that a djvu-mode can
> be added to auto-mode-alist.  I have not yet found the time to look at
> this more carefully.  But it should solve this problem.  In the
> meanwhile you could advice find-file-noselect as described in djvu.el,
> which should work in most cases, too.

Ah, I had seen that but forgot it.  Anyway, my little hack works well
enough for me until you add proper auto-mode-alist integration.

>> Another nice thing would be integration with the Emacs bookmark
>> facility (I haven't tried using the djvu bookmark facility, but it
>> looks like it's tied to djvu outlines, so orthogonal to Emacs
>> bookmarks).
>
> Djvu bookmarks are for efficiently moving within a djvu document,
> similar to how one can define bookmarks within a pdf document.  I have
> never used Emacs bookmarks, but it seems their main idea is different,
> moving around across different files.

My current need is simply to be at a specified position in a DJVU upon
visiting the file.  Probably both facilities do that but since I also
use Emacs bookmarks for other types of files, it's convenient to use
them for DJVU files as well.

>> I also like the way pdf-view-mode (and doc-view-mode) displays both
>> the current and total number of pages
>
> Yes, that makes sense; it will be part of the next update in GNU Elpa
> (implemented slightly different from your code).

Looking forward to it.

Steve Berman

RE: DJVU files and ImageMagick

[Roland Winkler]

On Tue Feb 5 2019 Drew Adams wrote:
> But the main idea of Emacs bookmarks is not about moving around
> across different files.  The main idea is just to record and
> return to a location.  Anywhere.  And a common use case is moving
> among locations in the same buffer (e.g. same file).

Thanks, I'll check how this can be turned into something useful for
djvu documents.