A few issues with thumbs.el

A few issues with thumbs.el

[Juanma Barranquero]

I've been testing thumbs.el on Windows, and I have a few issues:

  - Thumbs functions by default create jpeg thumbnails, but they don't
check whether the jpeg image support is loaded; that fails in
dynamic-loading environments. That is addressed in the patch below.

  - `thumbs-thumbname' makes a simple transformation on the image
filename to generate a thumbnail filename (taking out slashes and
changing spaces to underscores). That simple transformation fails on
Windows (it doesn't modify the ":" in drive:path, for example). I
could fix that, but instead of playing cat&mouse with unusual chars, I
think it'd be better to just make a digest of the name (with md5) and
get rid of the problem. That's addressed in the patch below, too.

  - When using `thumb', there's a crash on MSVC builds only; it
happens on jpeg_memory_src. I think the problem is unrelated to the
other day's troubles. I'll take a look at debugging it. Unaddressed.

  - The docstring of `thumbs-gensym' is wrong with respect to its
argument. But, instead of fixing the docstring, wouldn't be just
better to get rid of the argument? I mean, thumbs.el uses the function
without passing it, and I don't think there's much use outside of
thumbs.el for a `gensym'-alike that doesn't really add that much to
`gensym'. Or is there?

  - Other than being interactive, does `thumbs-kill-buffer' something
different that
   (quit-window t (selected-window))
  ?

                    /L/e/k/t/u



Index: lisp/ChangeLog
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/ChangeLog,v
retrieving revision 1.7592
diff -u -2 -r1.7592 ChangeLog
--- lisp/ChangeLog	25 May 2005 14:18:45 -0000	1.7592
+++ lisp/ChangeLog	25 May 2005 14:54:53 -0000
@@ -14,4 +14,8 @@
 	(thumbs-increment-image-size, thumbs-decrement-image-size):
  	Fix typos in docstrings.
+	(thumbs-thumbname): Use `md5' to safely encode all kinds of
+	filenames.
+	(thumbs-show-thumbs-list): Check for availability of the 'jpeg
+	image type.
 
  2005-05-24  Andre Spiegel  <spiegel@gnu.org>
Index: lisp/thumbs.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/thumbs.el,v
retrieving revision 1.11
diff -u -2 -r1.11 thumbs.el
--- lisp/thumbs.el	25 May 2005 14:08:33 -0000	1.11
+++ lisp/thumbs.el	25 May 2005 14:24:14 -0000
@@ -303,11 +303,5 @@
  (defun thumbs-thumbname (img)
   "Return a thumbnail name for the image IMG."
-  (concat thumbs-thumbsdir "/"
-	  (subst-char-in-string
-	   ?\  ?\_
-	   (apply
-	    'concat
-	    (split-string
-	     (expand-file-name img) "/")))))
+  (concat thumbs-thumbsdir "/" (md5 (expand-file-name img))))
 
  (defun thumbs-make-thumb (img)
@@ -379,6 +373,7 @@
 
  (defun thumbs-show-thumbs-list (L &optional buffer-name same-window)
-  (when (not (display-images-p))
-    (error "Images are not supported in this Emacs session"))
+  (unless (and (display-images-p)
+               (image-type-available-p 'jpeg))
+    (error "Required image type is not supported in this Emacs session"))
   (funcall (if same-window 'switch-to-buffer 'pop-to-buffer)
 	   (or buffer-name "*THUMB-View*"))

Re: A few issues with thumbs.el

[Stefan Monnier]

>   - `thumbs-thumbname' makes a simple transformation on the image
> filename to generate a thumbnail filename (taking out slashes and
> changing spaces to underscores). That simple transformation fails on
> Windows (it doesn't modify the ":" in drive:path, for example). I
> could fix that, but instead of playing cat&mouse with unusual chars, I
> think it'd be better to just make a digest of the name (with md5) and
> get rid of the problem. That's addressed in the patch below, too.

Maybe it could use convert-standard-file-name.


        Stefan

Re: A few issues with thumbs.el

[Juanma Barranquero]

> Maybe it could use convert-standard-file-name.

Something like that, you mean?

(defun thumbs-thumbname (img)
  "Return a thumbnail name for the image IMG."
  (convert-standard-filename
   (concat thumbs-thumbsdir "/" (expand-file-name img))))

It would work, but why should it be better than

(defun thumbs-thumbname (img)
 "Return a thumbnail name for the image IMG."
 (concat thumbs-thumbsdir "/" (md5 (expand-file-name img))))

?

(I have no bias, it'll work either way. I'm asking out of curiosity.)

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Stefan Monnier]

> It would work, but why should it be better than

> (defun thumbs-thumbname (img)
>  "Return a thumbnail name for the image IMG."
>  (concat thumbs-thumbsdir "/" (md5 (expand-file-name img))))

The resulting file name is at least somewhat meaningful.
Can be very handy when debugging a problem.  Maybe it could even be used for
other things (like globbing) in some cases.


        Stefan

Re: A few issues with thumbs.el

[Juanma Barranquero]

> The resulting file name is at least somewhat meaningful.

Well, yeah, but other than debugging (as you suggest), there's not
much use for the user accessing these files.

And I don't like the fact that the resulting files maintains the
extension of the originals (.tif, .jpg, .png, etc.), even when they're
all JPEG (the default thumbs.el output mode for convert).

Anyway, that's a bit of a bikeshed. I'll paint it "convert-standard-filename" :)

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Stefan Monnier]

> And I don't like the fact that the resulting files maintains the
> extension of the originals (.tif, .jpg, .png, etc.), even when they're
> all JPEG (the default thumbs.el output mode for convert).

Of course you need to add a .jpg extension, like foo_bar.gif.jpg.


        Stefan

Re: A few issues with thumbs.el

[Juanma Barranquero]

On second thought, this won't do:

  (defun thumbs-thumbname (img)
   "Return a thumbnail name for the image IMG."
   (convert-standard-filename
    (concat thumbs-thumbsdir "/" (expand-file-name img))))

because of directories, so it'll have to be:

  (defun thumbs-thumbname (img)
   "Return a thumbnail name for the image IMG."
   (convert-standard-filename
    (concat thumbs-thumbsdir "/"
            (subst-char-in-string
             ?\s ?\_
             (apply
              'concat
              (split-string
               (expand-file-name img) "/"))))))

which still strikes me as more fragile than using md5. The probability
of collision is higher, with the "_" vs " " and the fact that nothing
precludes c:/1/23.tif and c:/12/3.tif both existing.

I suppose I could transform the path separators to some other
character, but that seems like piling hack over hack.

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Stefan Monnier]

>   (defun thumbs-thumbname (img)
>    "Return a thumbnail name for the image IMG."
>    (convert-standard-filename
>     (concat thumbs-thumbsdir "/"
>             (subst-char-in-string
>              ?\s ?\_
>              (apply
>               'concat
>               (split-string
>                (expand-file-name img) "/"))))))

Why not just

  (defun thumbs-thumbname (img)
    "Return a thumbnail name for the image IMG."
    (concat thumbs-thumbsdir "/"
            (convert-standard-filename
             (subst-char-in-string ?/ ?\| (expand-file-name img)))))


-- Stefan

Re: A few issues with thumbs.el

[Juanma Barranquero]

> Why not just
> 
>   (defun thumbs-thumbname (img)
>     "Return a thumbnail name for the image IMG."
>     (concat thumbs-thumbsdir "/"
>             (convert-standard-filename
>              (subst-char-in-string ?/ ?\| (expand-file-name img)))))

ELISP> (thumbs-thumbname "C:/store/test/ycbcr-cat.tif")
"c:/usr/home/.emacs-thumb/C:!store!test!ycbcr-cat.tif"

which is not a valid name (the ":"). In fact, that was the error that
took me down this path.

So you need either to substitute several chars, maybe with a regexp
function (possibly filesystem-dependant), or add
`subst-char-in-string' layers. And "store!test", for example, is a
perfectly valid file or directory name, so ambiguity still exists.

Much, much easier to just use md5().

> Of course you need to add a .jpg extension, like foo_bar.gif.jpg.

I can add it just fine, but the original thumbs code doesn't do it. It
preserves the filename's extension. Should be irrelevant anyway.

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Stefan Monnier]

>> (defun thumbs-thumbname (img)
>> "Return a thumbnail name for the image IMG."
>> (concat thumbs-thumbsdir "/"
>> (convert-standard-filename
>> (subst-char-in-string ?/ ?\| (expand-file-name img)))))

ELISP> (thumbs-thumbname "C:/store/test/ycbcr-cat.tif")
> "c:/usr/home/.emacs-thumb/C:!store!test!ycbcr-cat.tif"

> which is not a valid name (the ":"). In fact, that was the error that
> took me down this path.

Oh, right.  So the docstring should be fixed to make it clear that the
argument should be an *absolute* file name.  And maybe we then need another
function "convert-standard-filename-element" for the non-directory
components of a file name.

Can someone confirm that the patch below is correct?


        Stefan


--- orig/lisp/files.el
+++ mod/lisp/files.el
@@ -514,6 +514,10 @@
 This means to guarantee valid names and perhaps to canonicalize
 certain patterns.
 
+FILENAME should be an absolute file name since the conversion rules
+sometimes vary depending on the position in the file name.  E.g. c:/foo
+is a valid DOS file name, but c:/bar/c:/foo is not.
+
 This function's standard definition is trivial; it just returns
 the argument.  However, on Windows and DOS, replace invalid
 characters.  On DOS, make sure to obey the 8.3 limitations.  On

Re: A few issues with thumbs.el

[Juanma Barranquero]

> Oh, right.  So the docstring should be fixed to make it clear that the
> argument should be an *absolute* file name.

Uh? convert-standard-filename works right with non-absolute paths:

(convert-standard-filename "file/test:.dat") => "file\\test!.dat"

It just handles an initial "X:" differently (on Windows):

(convert-standard-filename "c:file/test:.dat") => "c:file\\test!.dat"

I see nothing wrong with convert-standard-filename or its docstring.

AFAICS, the problem is that convert-standard-filename is intended to
transform a filename into another filename, suitable for the current
filesystem. But what you're trying in the code you proposed is using
convert-standard-filename to build the *name* part of a filename.

There's nothing anywhere that guarantees that a filename (with path
etc) is a valid name for a file. In MS-DOS/Windows you've got the ":",
in VMS the brackets, etc. When you concatenate thumbs-thumbsdir with
the output of convert-standard-filename, the result can perfectly be a
non-valid filename.

So you'll have to use convert-standard-filename, and then filter its
output to remove potentially dangerous characters, or, as I did in my
example, use convert-standard-filename as the last pass:

  (convert-standard-filename (concat thumbs-thumbsdir "/"
(subst-char-in-string [etc...])

I don't think the trouble is worth the tiny benefit that the filename
of the cached thumbnail is user-friendly on debugging. If someone has
to debug it he can modify thumbs-thumbname easily enough.

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Stefan Monnier]

> Uh? convert-standard-filename works right with non-absolute paths:
> (convert-standard-filename "file/test:.dat") => "file\\test!.dat"

It *sometimes* works right.  Which isn't very useful.

> It just handles an initial "X:" differently (on Windows):
> (convert-standard-filename "c:file/test:.dat") => "c:file\\test!.dat"

Indeed.  And if you want to place a config file named "a:b:c:d" in $HOME,
doing (expand-file-name (convert-standard-filename "a:b:c:d") "~")
will not return a valid answer on w32.
The result can only be trusted to be valid if the filename is absolute.


        Stefan

Re: A few issues with thumbs.el

[Juanma Barranquero]

> The result can only be trusted to be valid if the filename is absolute.

Yes, but that limitation is inherent in convert-standard-filename's
mission, unless you want to add it a second argument ABSOLUTE.

I mean, how would convert-standard-filename differentiate between
"x:/y" (a drive and path) and "x:/y" (a four-character name with
non-standard characters), when the goal of the function is *precisely*
to turn such non-valid names into valid ones?

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Stefan Monnier]

>> The result can only be trusted to be valid if the filename is absolute.
> Yes, but that limitation is inherent in convert-standard-filename's
> mission, unless you want to add it a second argument ABSOLUTE.

Indeed, but I think it's worth mentioning it in the docstring since I got
caught by it: it's a sensible limitation, but it's not an obvious one.


        Stefan

Re: A few issues with thumbs.el

[Juanma Barranquero]

> Indeed, but I think it's worth mentioning it in the docstring since I got
> caught by it: it's a sensible limitation, but it's not an obvious one.

Yes, you're right.

OTOH, to fix the function (other than modifying the docstring), I
think there's no need to make a new convert-standard-filename-element.
The interface of convert-standard-filename is not crowded, there's no
harm in adding a new argument.

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Richard Stallman]

    Indeed, but I think it's worth mentioning it in the docstring since I got
    caught by it: it's a sensible limitation, but it's not an obvious one.

Please add this info to the doc string if you want.

Re: A few issues with thumbs.el

[Juri Linkov]

> Much, much easier to just use md5().

Even though I like when thumbnail file names are composed from file
name parts (this allows to purge old thumbnails based on their names),
this method is not reliable, because thumbnails don't get updated when
image file contents changes.

AFAIK, GIMP uses md5 for naming thumbnail files.  It stores original
file names as comments in image files.  thumb.el could do the same
adding comments with original file names, e.g.:

/usr/bin/convert -comment "Description: Thumbnail of /home/user/images/image.jpg"
                 -sample 100x100 "/home/user/images/image.jpg"
                 "png:/home/user/.emacs-thumbs/b765f25b9ed0233aaefcd601e63d6c86.png"

BTW, in thumbs.el the condition for comparing geometry is currently
commented out:

	    ;;  This is not the right fix, but I don't understand
	    ;;  the external program or why it produces a geometry
	    ;;  unequal to the one requested -- rms.
;;;	    (not (equal (thumbs-file-size tn) thumbs-geometry))

The reason why the conversion program produces a different geometry is 
because it tries to maintain the aspect ratio of the original image.

One solution is to add the geometry string to thumbnail file names, e.g.

/home/user/.emacs-thumbs/100x100_b765f25b9ed0233aaefcd601e63d6c86.png

or to add the original geometry string to the thumbnail file comments:

/usr/bin/convert -comment "Geometry: 100x100" ...

with assumption that it is possible to extract comments in Emacs
from image files before comparing them with the current value of
thumbs-geometry.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: A few issues with thumbs.el

[Richard Stallman]

It would be much better for the user if the thumbnail file names are
based on something meaningful, such as the user's file name (or at
least part of it).  It is ok to add other info such as an md5 checksum
to make them unique, but please don't make them entirely meaningless.

Re: A few issues with thumbs.el

[Juanma Barranquero]

> It would be much better for the user if the thumbnail file names are
> based on something meaningful, such as the user's file name (or at
> least part of it).  It is ok to add other info such as an md5 checksum
> to make them unique, but please don't make them entirely meaningless.

I really don't understand why is that important; these files are not
for human consumption. But I don't really care one way or the other.
The real issue is not making them unique (although there's also a
small problem with that), but making them valid. As it is on CVS,
thumbs-thumbname does not produce valid MSDOS/Windows filenames.

I think the following approach can work, but still is could break on
non-Unix, non-Windows environments:

(defun thumbs-thumbname (img)
  "Return a thumbnail name for the image IMG."
  (convert-standard-filename
   (let ((filename (expand-file-name img)))
     (format "%s/%s-%x.jpg"
             thumbs-thumbsdir
             (subst-char-in-string
              ?\s ?\_
              (apply
               'concat
               (split-string filename "/")))
             (sxhash filename)))))

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Robert J. Chassell]

    I really don't understand why is that important; these files are not
    for human consumption.

It is very important:  I clean out directories once in a while.
Having names makes that possible with thumbnails.  Without the names,
I won't use thumbnails in Emacs.

You can talk about deleting images that you see, but that is not how I
operate.  Presumably, some people do, but that is neither here nor
there.

-- 
    Robert J. Chassell                         
    bob@rattlesnake.com                         GnuPG Key ID: 004B4AC8
    http://www.rattlesnake.com                  http://www.teak.cc

Re: A few issues with thumbs.el

[Juanma Barranquero]

> It is very important:  I clean out directories once in a while.
> Having names makes that possible with thumbnails.  Without the names,
> I won't use thumbnails in Emacs.

Are you saying that if you have a .emacs-thumbs directory with a few
hundreds files with names like "c!tmpimagemyfile.png.jpg" and
"d!storemypicsdog734.tif.jpg" you're gonna delete them selectively
instead of simply deleting the whole bunch of them and let them be
recreated on demand? Curious.

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Robert J. Chassell]

   Are you saying that if you have a .emacs-thumbs directory with a few
   hundreds files with names like "c!tmpimagemyfile.png.jpg" and
   "d!storemypicsdog734.tif.jpg" you're gonna delete them selectively

No, I am saying that if I have pictures like

    Barnacle-Bill-high-res.jpg.jpg
    Bob-by-airplane.jpg.jpg
    cottage-downstairs.png.jpg

I am going to delete them selectively.  For example,
Barnacle-Bill-high-res.jpg.jpg is a picture taken several years ago of
a named rock on Mars.  I might want to get rid of its thumbnail.

When the thumbnails are

    dscf0002.jpg.jpg
    dscf0003.jpg.jpg
    dscf0004.jpg.jpg

then I delete them all.  I do not want to spend the minutes it takes
to recreate thumbnails for the images in the directory for which I
mainly need thumbnails.  It takes too long.

-- 
    Robert J. Chassell                         
    bob@rattlesnake.com                         GnuPG Key ID: 004B4AC8
    http://www.rattlesnake.com                  http://www.teak.cc

Re: A few issues with thumbs.el

[Juanma Barranquero]

> No, I am saying that if I have pictures like
> 
>     Barnacle-Bill-high-res.jpg.jpg
>     Bob-by-airplane.jpg.jpg
>     cottage-downstairs.png.jpg
> 
> I am going to delete them selectively.

Sure. It's just that the current code wouldn't create these files, at
least not on Windows (and I doubt it would on Unix or GNU/Linux). They
would me more like

 c:\\home\\.emacs-thumbs\\c!documents_and_settingsmy_user_namemy_documentsmy_picturesastronomyBarnacle-Bill-high-res.jpg.jpg"

which makes discriminating among a bunch of them quite less easy.

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Robert J. Chassell]

    ... They would me more like

    c:\\home\\.emacs-thumbs\\c!documents_and_settingsmy_user_namemy_documentsmy_picturesastronomyBarnacle-Bill-high-res.jpg.jpg"

   which makes discriminating among a bunch of them quite less easy.

Yes, but still it is easier than a pure number.  I think your idea of
combining an md5 sum with the name is fine.  I will simply ignore the
number and look at the name.  I understand the need for the number.
You have a good solution in mind.

-- 
    Robert J. Chassell                         
    bob@rattlesnake.com                         GnuPG Key ID: 004B4AC8
    http://www.rattlesnake.com                  http://www.teak.cc

Re: A few issues with thumbs.el

[Richard Stallman]

    I really don't understand why is that important; these files are not
    for human consumption.

It is a mistake to think about any files that way.
Files are always user-visible.

In general, the only time something is really not visible to users
is when it is internal to one program.

Re: A few issues with thumbs.el

[Juanma Barranquero]

> In general, the only time something is really not visible to users
> is when it is internal to one program.

I suppose that is one way to see it, yes. But using md5 or other
checksum/hash ways to encode names of temporary files is hardly new
(though I suppose you'll say it is not new, but it *is* wrong
nonetheless :)

But the point is moot. Does anyone object to the version of
thumbs-thumbname I proposed a few messages ago, where the resulting
filename is very much like the one it generates now, but valid and
with a sxhash checksum added to it to make collisions less likely?

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Juanma Barranquero]

> Even though I like when thumbnail file names are composed from file
> name parts (this allows to purge old thumbnails based on their names),
> this method is not reliable, because thumbnails don't get updated when
> image file contents changes.

You're right, the "correct" way would be to make an md5 or sha1 of the
file contents (not the filename) and add that to the thumbname.

> AFAIK, GIMP uses md5 for naming thumbnail files.  It stores original
> file names as comments in image files.  thumb.el could do the same
> adding comments with original file names, e.g.:

I'd like that, but the most prevalent opinion has been that people
wants to have the filename directly visible in the thumbname, so
that's what I've finally implemented.

> One solution is to add the geometry string to thumbnail file names, e.g.
> 
> /home/user/.emacs-thumbs/100x100_b765f25b9ed0233aaefcd601e63d6c86.png
> 
> or to add the original geometry string to the thumbnail file comments:
> 
> /usr/bin/convert -comment "Geometry: 100x100" ...

Comments would perhaps be better; adding more and more information to
the thumbnames makes them unwieldy. In fact, I've used a sxhash and
not md5 to make thumbnames more unique because they are already quite
large (as they include a slightly mangled full path).

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Juri Linkov]

>> AFAIK, GIMP uses md5 for naming thumbnail files.  It stores original
>> file names as comments in image files.  thumb.el could do the same
>> adding comments with original file names, e.g.:
>
> I'd like that, but the most prevalent opinion has been that people
> wants to have the filename directly visible in the thumbname, so
> that's what I've finally implemented.

Maybe just a file name without the directory part is enough for people
to recognize the thumbnail file?  This makes thumbnail filenames much
shorter, so they fit into 80-character wide windows (currently thumbnail
filenames are so long that they are truncated in dired buffers and
the non-directory part is not visible without horizontal scrolling).

How about the following template?

(format "%s/%s-%x.jpg"
        thumbs-thumbsdir
        (file-name-nondirectory filename)
        (sxhash contents))

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: A few issues with thumbs.el

[Juanma Barranquero]

> How about the following template?
> 
> (format "%s/%s-%x.jpg"
>         thumbs-thumbsdir
>         (file-name-nondirectory filename)
>         (sxhash contents))

A problem is that (sxhash contents) will be slower; I have no issue
with that, but perhaps someone will. Other than that, I'm OK with any
fix that produces valid filenames and minimizes collisions.

See what other people think before committing the change, though.
Small decisions often bring big discussions :)

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[David Kastrup]

Juanma Barranquero <lekktu@gmail.com> writes:

>> How about the following template?
>> 
>> (format "%s/%s-%x.jpg"
>>         thumbs-thumbsdir
>>         (file-name-nondirectory filename)
>>         (sxhash contents))
>
> A problem is that (sxhash contents) will be slower; I have no issue
> with that, but perhaps someone will. Other than that, I'm OK with any
> fix that produces valid filenames and minimizes collisions.

I'd rather hash the file modification date rather than the contents.
What's good enough for `make'...

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: A few issues with thumbs.el

[Stefan Monnier]

A few questions about thumbs.el:

- It has some non-negligible actions at the top-level, which is contrary to
  the conventions (the reason for those conventions is that the file may get
  loaded without any intention to actually use the mode.  E.g. it may be
  loaded by Custom, or by the byte-compiler, ...).
  These are bugs and should be fixed.  I'm thinking of:

   ;; Make sure auto-image-file-mode is ON.
   (auto-image-file-mode t)
   
   ;; Create the thumbs directory if it does not exists.
   (setq thumbs-thumbsdir (expand-file-name thumbs-thumbsdir))
   
   (when (not (file-directory-p thumbs-thumbsdir))
     (progn
       (make-directory thumbs-thumbsdir)
       (message "Creating thumbnails directory")))
   
   (when thumbs-thumbsdir-auto-clean
     (thumbs-cleanup-thumbsdir))

- I wish there was a way to put the thumbs directly next to the original
  image files.  E.g. instead of ~/.emacs-thumbs/foo!bar!baz!toto!NNNN.jpg
  I'd like to use ~/foo/bar/baz/.emacs-thumbs/toto_NNNN.jpg.


-- Stefan

Re: A few issues with thumbs.el

[Juanma Barranquero]

> A few questions about thumbs.el:

First: I'm not the author, nor the maintainer; I just got interested
on thumbs.el because I'm working on the image support on Windows.

>   These are bugs and should be fixed.  I'm thinking of:

I agree.

> - I wish there was a way to put the thumbs directly next to the original
>   image files.  E.g. instead of ~/.emacs-thumbs/foo!bar!baz!toto!NNNN.jpg
>   I'd like to use ~/foo/bar/baz/.emacs-thumbs/toto_NNNN.jpg.

Ah. A .emacs-thumb subdirectory on each directory with images. That'd
be nice, but it would make cleaning old thumbs harder, wouldn't it?

Anyway, if the author doesn't fix these issues, I'll take a look at
them in a few weeks.

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Richard Stallman]

I think we will want to replace thumbs.el with tumme.el.
The developer of tumme.el is looking at adding in
all the features of thumbs.el so that nothing will be lost
through the change.

      the conventions (the reason for those conventions is that the file may get
      loaded without any intention to actually use the mode.  E.g. it may be
      loaded by Custom, or by the byte-compiler, ...).
      These are bugs and should be fixed.  I'm thinking of:

       ;; Make sure auto-image-file-mode is ON.
       (auto-image-file-mode t)

There is no reason now to use auto-image-file-mode.
I think Image mode is much better.

Re: A few issues with thumbs.el

[Juanma Barranquero]

> I think we will want to replace thumbs.el with tumme.el.
> The developer of tumme.el is looking at adding in
> all the features of thumbs.el so that nothing will be lost
> through the change.

Hmm. thumbs.el is in Emacs since 2004-02-28, and so it's not been
distributed in any official Emacs release, so if we're going to switch
to tumme.el, it'd be better to do it before 22.1.

OTOH, tumme.el is very nice, but it is still a little "green" (both in
terms of features and in following Emacs conventions for modules).

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Juanma Barranquero]

On 5/30/05, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

>   These are bugs and should be fixed.  I'm thinking of:
> 
>    ;; Make sure auto-image-file-mode is ON.
>    (auto-image-file-mode t)

Removed.

> 
>    ;; Create the thumbs directory if it does not exists.
>    (setq thumbs-thumbsdir (expand-file-name thumbs-thumbsdir))
> 
>    (when (not (file-directory-p thumbs-thumbsdir))
>      (progn
>        (make-directory thumbs-thumbsdir)
>        (message "Creating thumbnails directory")))

I've added a new function to return the thumbnails dir, creating it if needed.

Additionally, I've had to change a few more things, like
thumbs-temp-file, which got created and didn't update when
thumbs-thumbdir changed; now it is a function. I've also turned
thumbs-gensym into a CL gensym clone; there was no point in having a
different default prefix or anything (no one in his/her sane mind is
gonna load thumbs.el just in order to have gensym).

>    (when thumbs-thumbsdir-auto-clean
>      (thumbs-cleanup-thumbsdir))

This is the only extant top-level action. The question is, what to do
with the thumbsdir cleanup? It could be an interactive command issued
by the user (not pretty, but not entirely unreasonable), or it could
be automatically triggered by the functions that create new
thumbnails.  If so, in order to be efficient, it should be called
*after* creating the thumbnails, passing it a list of recently-created
thumbnails that it should preserve; otherwise, when the directory has
reached the limit, every single thumbnail creation would trigger
deletion of old thumbnails, which, in directories with many images,
would mean that a lot of thumbnails are deleted just to be recreated
immediately.

Alternatively, the cleanup could be triggered periodically, via timers
or by any other means.

The problem with both alternatives is that there will be circumstances
when the thumbnails directory will be bigger (perhaps much bigger)
than `thumbs-thumbsdir-max-size'; but this is no different of what
happens right now.

Another question. Are things like:

(defcustom thumbs-conversion-program
  ...
  (or (executable-find "convert")
   ...

kosher? I ask because this is, after all, a top-level action executed
on loading.

> - I wish there was a way to put the thumbs directly next to the original
>   image files.  E.g. instead of ~/.emacs-thumbs/foo!bar!baz!toto!NNNN.jpg
>   I'd like to use ~/foo/bar/baz/.emacs-thumbs/toto_NNNN.jpg.

Hmm. Is NNNN.jpg an image on toto/, or toto_NNNN.jpg an image on baz/?

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Stefan Monnier]

>> ;; Make sure auto-image-file-mode is ON.
>> (auto-image-file-mode t)
> Removed.

Thanks.

>> (when thumbs-thumbsdir-auto-clean
>> (thumbs-cleanup-thumbsdir))

> This is the only extant top-level action. The question is, what to do
> with the thumbsdir cleanup? It could be an interactive command issued
> by the user (not pretty, but not entirely unreasonable), or it could
> be automatically triggered by the functions that create new
> thumbnails.  If so, in order to be efficient, it should be called
> *after* creating the thumbnails, passing it a list of recently-created
> thumbnails that it should preserve; otherwise, when the directory has
> reached the limit, every single thumbnail creation would trigger
> deletion of old thumbnails, which, in directories with many images,
> would mean that a lot of thumbnails are deleted just to be recreated
> immediately.

> Alternatively, the cleanup could be triggered periodically, via timers
> or by any other means.

> The problem with both alternatives is that there will be circumstances
> when the thumbnails directory will be bigger (perhaps much bigger)
> than `thumbs-thumbsdir-max-size'; but this is no different of what
> happens right now.

Indeed, the delete-upon-load is also a heuristic and it's not necessarily
better than any other.  I think anything is OK, as long as it's run at least
once per Emacs session where you use thumbs without being run too often to
slow everything down.

> Another question. Are things like:

> (defcustom thumbs-conversion-program
>   ...
>   (or (executable-find "convert")
>    ...

> kosher? I ask because this is, after all, a top-level action executed
> on loading.

executable-find is not expected to modify any state: it doesn't
create/modify/delete any file for exemple.  It's more or less
a "pure" function.  Some files even run programs upon load (typically
things like "ispell --version"), and while it's often sub-optimal, it's
still OK as long as the program itself won't change any state either.

>> - I wish there was a way to put the thumbs directly next to the original
>> image files.  E.g. instead of ~/.emacs-thumbs/foo!bar!baz!toto!NNNN.jpg
>> I'd like to use ~/foo/bar/baz/.emacs-thumbs/toto_NNNN.jpg.

> Hmm. Is NNNN.jpg an image on toto/, or toto_NNNN.jpg an image on baz/?

toto_NNNN.jpg would be the thumb for (e.g.) baz/toto.gif.
The NNNN is in case you want to add a hash or counter.


        Stefan

Re: A few issues with thumbs.el

[Juanma Barranquero]

> Indeed, the delete-upon-load is also a heuristic and it's not necessarily
> better than any other.  I think anything is OK, as long as it's run at least
> once per Emacs session where you use thumbs without being run too often to
> slow everything down.

OK, I'll do a little thinking about that.

> It's more or less a "pure" function.

That's what I thought. Thanks.

> Some files even run programs upon load (typically
> things like "ispell --version"), and while it's often sub-optimal, it's
> still OK as long as the program itself won't change any state either.

Yeah. In fact I want to do that to detect whether the convert.exe in
the path is the ImageMagick's or Windows' one (alternatively, I'm
thinking of temporarily removing %windir%/* directories from exec-path
before trying to find convert.exe).

> toto_NNNN.jpg would be the thumb for (e.g.) baz/toto.gif.
> The NNNN is in case you want to add a hash or counter.

Ah, OK. 

I have a few fixes for thumbs.el (with my latest patches it could fail
to create a valid temporary file name if thumbs-thumbdir or
thumbs-temp-dir do not end in slash), but I'll wait till I've given
consideration to these issues.

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Stefan Monnier]

>> Some files even run programs upon load (typically
>> things like "ispell --version"), and while it's often sub-optimal, it's
>> still OK as long as the program itself won't change any state either.

> Yeah. In fact I want to do that to detect whether the convert.exe in
> the path is the ImageMagick's or Windows' one (alternatively, I'm
> thinking of temporarily removing %windir%/* directories from exec-path
> before trying to find convert.exe).

I'm not sure what the convert.exe that comes with Windows does, but I assume
it's of no use to thumbs.el.  In such a case running the program might not
be that useful since all it will allow us to do is to tell the user that his
"convert" isn't the one we need.  We could do that just as well when we
actually really have to run it (as part of a generic error-handling or
sanity-checking code that ensures that convert.exe indeed produced the file
we wanted).

OTOH giving preference to a convert.exe which is not in %windir%/* might
indeed help make the code run in more circumstances, so it's probably a good
idea (although it may still want to use a convert.exe found in %windir%/* if
it couldn't find any other).


        Stefan

Re: A few issues with thumbs.el

[Juanma Barranquero]

> I'm not sure what the convert.exe that comes with Windows does, but I assume
> it's of no use to thumbs.el.

No. It does convert FAT volumes to NTFS.

> OTOH giving preference to a convert.exe which is not in %windir%/* might
> indeed help make the code run in more circumstances, so it's probably a good
> idea

That's my first choice.

> (although it may still want to use a convert.exe found in %windir%/* if
> it couldn't find any other).

Hmm. Not sure about this one. I think the probability of having a
ImageMagick convert.exe in %windir% is very low on W9X/Me and zero, to
all practical effects, on W2K/XP (not sure about NT). The ImageMagick
distributions for Windows (there's a "standard" binary tarball and a
Cygwin package) do not install executables on %windir%.

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Stefan Monnier]

>> (although it may still want to use a convert.exe found in %windir%/* if
>> it couldn't find any other).

> Hmm. Not sure about this one. I think the probability of having a
> ImageMagick convert.exe in %windir% is very low on W9X/Me and zero, to
> all practical effects, on W2K/XP (not sure about NT). The ImageMagick
> distributions for Windows (there's a "standard" binary tarball and a
> Cygwin package) do not install executables on %windir%.

I'll trust you on that one.  I actually had no idea, which is why I said
"may".


        Stefan

Re: A few issues with thumbs.el

[Mathias Dahl]

Juanma Barranquero <lekktu@gmail.com> writes:

> Yeah. In fact I want to do that to detect whether the convert.exe in
> the path is the ImageMagick's or Windows' one (alternatively, I'm
> thinking of temporarily removing %windir%/* directories from
> exec-path before trying to find convert.exe).

I don't understand why we couldn't just leave this to the user to
configure this, instead of trying to have more or less complex logic,
trying to figure this out automatically. `find-dired' has the same
problem in that it will find (haha) the wrong "find" on w32. It also
has more or less hard-coded syntax in it.

Juanma, you know how tumme.el currently does this; that works quite
well, does it not?

For others, this is how tumme currently handle this:

(defcustom tumme-cmd-create-thumbnail-program
  "convert"
  "*Executable used to create thumbnail. Used together with
`tumme-cmd-create-thumbnail-options'."
  :type 'string
  :group 'tumme)

...

(defcustom tumme-cmd-create-thumbnail-options
  "%e -size %sx%s \"%f\" -resize %sx%s +profile \"*\" jpeg:\"%t\""
  "*Format of command used to create thumbnail image. Available
options are %e which is replaced by
`tumme-cmd-create-thumbnail-program', %s which is replaced by
`tumme-thumb-size', %f which is replaced by the file name of the
original image and %t which is replaced by the file name of the
thumbnail file."
  :type 'string
  :group 'tumme)

...

(defun tumme-create-thumb (file)
  "Using the convert program, create thumbnail image for FILE and
place it in `tumme-dir'"
  (let* ((size (int-to-string tumme-thumb-size))
         (command
          (format-spec
           tumme-cmd-create-thumbnail-options
           (list
            (cons ?e tumme-cmd-create-thumbnail-program)
            (cons ?s size)
            (cons ?f file)
            (cons ?t (expand-file-name (tumme-thumb-name file)))))))
    (if (file-exists-p tumme-dir)
        (if (not (file-directory-p tumme-dir))
            (error "%s exists but is not a directory!" tumme-dir))
      (make-directory tumme-dir))
    (shell-command command nil)))

/Mathias

Re: A few issues with thumbs.el

[Juanma Barranquero]

> I don't understand why we couldn't just leave this to the user to
> configure this, instead of trying to have more or less complex logic,
> trying to figure this out automatically. `find-dired' has the same
> problem in that it will find (haha) the wrong "find" on w32. It also
> has more or less hard-coded syntax in it.

Well, on one hand convert, unlike find, is potentially dangerous if
used by accident; however, I agree that the chance of executing it in
such a way that it "works" and destroys user data is abysmally small.

My main argument (but I won't argue much :-) is that it seems a bit
silly (both for convert.exe and find.exe) to set up a variable with a
value that we *know* it is wrong. I'd rather have *-convert-program
set to nil that to "C:/windows/convert.exe". ImageMagick's convert.exe
is simply *not* going to end in %windir% for any noticeable percentage
of Windows users.

> Juanma, you know how tumme.el currently does this; that works quite
> well, does it not?

Manual configuration is good, of course. More good that wrong
automatic configuration, less good that correct automatic
configuration :-)

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Mathias Dahl]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > Alternatively, the cleanup could be triggered periodically, via
> > timers or by any other means.
> 
> > The problem with both alternatives is that there will be
> > circumstances when the thumbnails directory will be bigger
> > (perhaps much bigger) than `thumbs-thumbsdir-max-size'; but this
> > is no different of what happens right now.
> 
> Indeed, the delete-upon-load is also a heuristic and it's not
> necessarily better than any other.  I think anything is OK, as long
> as it's run at least once per Emacs session where you use thumbs
> without being run too often to slow everything down.

I am merging this into tumme.el right now. How about something simple
like using a defvar to store information about whether the clean-up
function has been called in this session or not?

After each call to `tumme-display-thumbs', check that variable, and if
nil, ask the user if it is OK to do a clean up (or really check if a
clean-up is necessary). We can provide options like "Yes, please
clen-up", "No, ask me later", or "No, and don't bother me again this
session" or something like that. Together with a custom variable that
determines if we should use cleaning at all, this should suit most
user's needs, right?

What do you say?

/Mathias

Re: A few issues with thumbs.el

[Juanma Barranquero]

> After each call to `tumme-display-thumbs', check that variable, and if
> nil, ask the user if it is OK to do a clean up (or really check if a
> clean-up is necessary). We can provide options like "Yes, please
> clen-up", "No, ask me later", or "No, and don't bother me again this
> session" or something like that. Together with a custom variable that
> determines if we should use cleaning at all, this should suit most
> user's needs, right?

You'd need a user option (settable to 'never, 'once-per-session,
'always and nil for "ask the user"), and an internal variable to
record when the cleanup ran last time, wouldn't you?

But, if you implement Stefan's per-directory thumbnails dir, every
directory would/should have its own set of configuration options, I
think. Perhaps not for *when* to run the cleanup, but I can well
imagine that U:\myimages\.thumbsdir (in my USB pendrive) doesn't need
the same size settings than "C:\my documents\my images\.thumbsdir"...

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Mathias Dahl]

Juanma Barranquero <lekktu@gmail.com> writes:

>> After each call to `tumme-display-thumbs', check that variable, and if
>> nil, ask the user if it is OK to do a clean up (or really check if a
>> clean-up is necessary). We can provide options like "Yes, please
>> clen-up", "No, ask me later", or "No, and don't bother me again this
>> session" or something like that. Together with a custom variable that
>> determines if we should use cleaning at all, this should suit most
>> user's needs, right?
>
> You'd need a user option (settable to 'never, 'once-per-session,
> 'always and nil for "ask the user"), and an internal variable to
> record when the cleanup ran last time, wouldn't you?

Something like that, yes.

> But, if you implement Stefan's per-directory thumbnails dir, every
> directory would/should have its own set of configuration options, I
> think. Perhaps not for *when* to run the cleanup, but I can well
> imagine that U:\myimages\.thumbsdir (in my USB pendrive) doesn't
> need the same size settings than "C:\my documents\my
> images\.thumbsdir"...

Sounds messy to me.

Re: A few issues with thumbs.el

[Juanma Barranquero]

> Sounds messy to me.

I'm not particularly for or against the local thumbnail directories
that Stefan proposes; but if implemented, they wouldn't be useful to
me without local settings.

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Richard Stallman]

Please note that we are likely to replace thumbs.el with tumme.el,
which seems to be better in general.

Re: A few issues with thumbs.el

[Juanma Barranquero]

> Please note that we are likely to replace thumbs.el with tumme.el,
> which seems to be better in general.

I know, but meanwhile I prefer to have a working thumbs.el.

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Mathias Dahl]

Juanma Barranquero <lekktu@gmail.com> writes:

>> Please note that we are likely to replace thumbs.el with tumme.el,
>> which seems to be better in general.
>
> I know, but meanwhile I prefer to have a working thumbs.el.

While you are doing fixes in thumbs.el, you wouldn't care for merging all of
tumme.el into it, would you? ... :)

Seriously, I still have functionality to merge / recreate from thumbs.el into
tumme.el to make it have same or similar functionality. While doing that I have
been thinking if there are things in thumbs.el that we do not even
want. Examples of things I personally find somewhat "unecessary" is the setroot
("wallpaper") functionality. As a tumme.el user myself (yes, I created it for
myself so of course I use it... :) I would have done that from dired using a
simple dired-do-shell-command.

What do others say?

/Mathias

Re: A few issues with thumbs.el

[Juanma Barranquero]

> While you are doing fixes in thumbs.el, you wouldn't care for merging all of
> tumme.el into it, would you? ... :)

Not at all, but then, if tumme.el is really going to be integrated
into Emacs soonish, I'll stop changing thumbs.el and concentrate on
tumme.el.

> What do others say?

Yes, I agree there are things in thumbs.el which don't seem generic
enough to be worth the effort of maintaining them in an Emacs package.

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Juanma Barranquero]

OK, I've installed a few patches for thumbs.el that fix suboptimal
code in my previous change (mostly, situations where a
non-slash-finished directory could be accidentally concatenated to a
file name).

I'll stop now modifying thumbs.el till a decision is reached about
tumme.el. Unimplemented so far: removing the thumbsdir cleanup from
the top level, and the customization of the thumb dir so it could be
along the image directories.

-- 
                    /L/e/k/t/u

Re: A few issues with thumbs.el

[Mathias Dahl]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
 
> - I wish there was a way to put the thumbs directly next to the original
>   image files.  E.g. instead of ~/.emacs-thumbs/foo!bar!baz!toto!NNNN.jpg
>   I'd like to use ~/foo/bar/baz/.emacs-thumbs/toto_NNNN.jpg.

What are the benefits of doing this? I think it would litter the
system with extra directories. I can see how it could solve the
uniqueness-problem, not needing to add a hash anymore, but I don't
know how big a problem that is (see discussion below).

About thumbnail file naming:

Currently, in tumme.el, I use a very simple way for creating the
thumbnail file name:

my_picture.jpg => ~/.tumme/my_picture.thumb.jpg

my_other_picture.png => ~/.tumme/my_other_picture.thumb.jpg

It works on my system because I have seen to it, by adding the images'
EXIF DateTime-value to it, that all my images have unique file names,
but I can see that is not likely on all user's setups.

The potential for collisions will vary from one system to the other;
some might have unique names like me, some might have a structure
where they rely on the path to make the files unique.

I think adding a hash at the end of the file name would be OK.

Something like this, which is a slightly changed version of Juri's
suggestion would be nice:

(format "%s/%s-%x.jpg"
        thumbnail-directory
        (file-name-nondirectory filename)
        (sxhash filename))

The above would not work well for those that have many images with the
same file name though, if they need to identify their images'
thumbnail files.

Btw, does anyone else agree with me that using something else than
JPEG as thumbnail image format is a bad idea? I have tried using PNG
but the files gets much larger.

/Mathias

Re: A few issues with thumbs.el

[Henrik Enberg]

Mathias Dahl <brakjoller@gmail.com> writes:

> About thumbnail file naming:

Have you seen the thumbnail-spec, which is apparently poised to become a
freedesktop.org standard?  It seems to allow usage of the same
thumbnails by several programs.

<http://jens.triq.net/thumbnail-spec/>

Re: A few issues with thumbs.el

[Mathias Dahl]

Henrik Enberg <henrik.enberg@telia.com> writes:

> Have you seen the thumbnail-spec, which is apparently poised to become a
> freedesktop.org standard?  It seems to allow usage of the same
> thumbnails by several programs.
> 
> <http://jens.triq.net/thumbnail-spec/>

I have now... :)

Seems quite good although I don't like the idea of using PNG-files as
their size is much larger than using JPEG.

The file names are named according to the absolute URI's MD5 value,
which most in this thread seems to dislike (including myself).

/Mathias

Re: A few issues with thumbs.el

[Paul Pogonyshev]

Mathias Dahl wrote:
> Henrik Enberg <henrik.enberg@telia.com> writes:
> > Have you seen the thumbnail-spec, which is apparently poised to become a
> > freedesktop.org standard?  It seems to allow usage of the same
> > thumbnails by several programs.
> >
> > <http://jens.triq.net/thumbnail-spec/>
>
> I have now... :)
>
> Seems quite good although I don't like the idea of using PNG-files as
> their size is much larger than using JPEG.
>
> The file names are named according to the absolute URI's MD5 value,
> which most in this thread seems to dislike (including myself).

In this case you should bug the specification author/maintainer.  If this
gets approved as a standard, I think it will be very counter-productive to
use a different scheme with Emacs.

Paul

Re: A few issues with thumbs.el

[Juri Linkov]

>> Have you seen the thumbnail-spec, which is apparently poised to become a
>> freedesktop.org standard?  It seems to allow usage of the same
>> thumbnails by several programs.
>> 
>> <http://jens.triq.net/thumbnail-spec/>
>
> I have now... :)
>
> Seems quite good although I don't like the idea of using PNG-files as
> their size is much larger than using JPEG.

PNG is more optimal for small images like thumbnails.

> The file names are named according to the absolute URI's MD5 value,
> which most in this thread seems to dislike (including myself).

This specification is _already_ implemented in Gimp.  Since Gimp is
the primary GNU image manipulation program, this is the _de-facto_
GNU standard for creating thumbnails.

Gimp users already have the directory ~/.thumbnails.  Some time
ago thumbs.el was added to the Emacs CVS tree which uses the
~/.emacs-thumbs directory for thumbnails.  And now you've created
a new package that uses yet another directory ~/.tumme.
All these three directories are used for the same purpose.
This is a waste of user space.

The above-mentioned specification is well thought out, and the list of
advantages of this approach is convincing.  So the best thing for us
to do now is just to follow this standard and to share the same
standard thumbnail directory with other GNU programs.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: A few issues with thumbs.el

[Mathias Dahl]

Juri Linkov <juri@jurta.org> writes:

> PNG is more optimal for small images like thumbnails.

In what way? The tests I did showed that thumbnails of size 100x100
was several times larger.

Re: A few issues with thumbs.el

[Mathias Dahl]

Juri Linkov <juri@jurta.org> writes:

> The above-mentioned specification is well thought out, and the list
> of advantages of this approach is convincing.  So the best thing for
> us to do now is just to follow this standard and to share the same
> standard thumbnail directory with other GNU programs.

Point taken. I will put this on my todo-list for tumme.el. It should
not be that hard to implement.

The onlu doubt I have is about those "local" thumbnail directories. I
did not understand when one was supposed to created them.

Re: A few issues with thumbs.el

[Daniel Brockman]

Mathias Dahl <brakjoller@gmail.com> writes:

> Juri Linkov <juri@jurta.org> writes:
>
> The onlu doubt I have is about those "local" thumbnail directories. I
> did not understand when one was supposed to created them.

As I understood it, your program is not supposed to create them unless
the user specifically requests that it do.  I don't think that feature
is absolutely necessary, so I'd say you can leave it out if you want.

But you should probably read those files if the `.thumblocal'
directory is already there, as described by the specification.

-- 
Daniel Brockman <daniel@brockman.se>

   ``Why fix an old bug if you can write three new ones
     in the same time?'' --- David Kastrup (on emacs-devel)

Re: A few issues with thumbs.el

[Mathias Dahl]

Mathias Dahl <brakjoller@gmail.com> writes:


>> The above-mentioned specification is well thought out, and the list
>> of advantages of this approach is convincing.  So the best thing for
>> us to do now is just to follow this standard and to share the same
>> standard thumbnail directory with other GNU programs.
>
> Point taken. I will put this on my todo-list for tumme.el. It should
> not be that hard to implement.

I did some investigations yesterday and found that there is something
I'm not sure how we could solve in a good way in Emacs: the tEXt
chunks (text annotations) in PNG files.

The ImageMagick package contains a utility called `identify' that can
read these values in the following way:

$ identify -verbose my_file.png | grep Thumb::

So, as both tumme.el and thumbs.el already use ImageMagick, although
only the convert and mogrify tools (tumme.el could use only convert
too if I wanted it too, but mogrify simplifies things a bit), I see no
problem in how to read those annotations.

The problem is writing them. After searching the net for a while I
have found two ways to do it, using external tools:

* pngcrush - a command line tool for compressing PNG to make them
smaller, but which also can add tEXt chunks to PNG files.

* a small perl-script, png-tEXt.pl, which sole purpose is to add text
  annotations to PNG files.

I'm not sure about the licence for them, but they seem free enough (at
least gratis) for use to use them.

What I do not like is involving yet another utility and I have
therefore a question:

Would it be possible to (and would we want to, Emacs is after all
mostly a text-processing tool, not a image manipulation app), in the
long run, learn Emacs to read and write these annotations? If I
understand it correctly pngcrush uses libpng, and that is what Emacs
uses too, right?

The subject of thumbnails seems to be sensitive, so I'll concentrate
on fixing other things in tumme.el, in the meantime... :)

Re: A few issues with thumbs.el

[Kim F. Storm]

Mathias Dahl <brakjoller@gmail.com> writes:

> Seems quite good although I don't like the idea of using PNG-files as
> their size is much larger than using JPEG.

As long as they are smaller than the block size on your file system,
it probably doesn't matter...

>
> The file names are named according to the absolute URI's MD5 value,
> which most in this thread seems to dislike (including myself).

Is there really such a thing as an absolute URI?

E.g. if the images are on a server somewhere -- the file name on
windoze may be h:\x\y\z.png while on GNU the file name may be
/mnt/images/z.png.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: A few issues with thumbs.el

[Mathias Dahl]

storm@cua.dk (Kim F. Storm) writes:

>> Seems quite good although I don't like the idea of using PNG-files as
>> their size is much larger than using JPEG.
>
> As long as they are smaller than the block size on your file system,
> it probably doesn't matter...

Good point. Anyway, PNG is the format suggested in the spec and one
could guess that they did some investigation on pros and cons of
various image formats before deciding on PNG...


>> The file names are named according to the absolute URI's MD5 value,
>> which most in this thread seems to dislike (including myself).
>
> Is there really such a thing as an absolute URI?
>
> E.g. if the images are on a server somewhere -- the file name on
> windoze may be h:\x\y\z.png while on GNU the file name may be
> /mnt/images/z.png.

Sorry, that was probably a sloppy description from me. What they
suggest is to generate the URL/URI (which is it?) to the file, so:

/home/mathias/pics/my_pic_01.jpg

becomes

file:///home/mathias/pics/my_pic_01.jpg

After that they compute the MD5 sum for this value and use that as
thumbnail file name. Also, some information is added to the PNG file
(original file name and some more stuff). Read the full spec for more
information, I better not try to explain it myself again... :)

Re: A few issues with thumbs.el

[Daniel Brockman]

Mathias Dahl <brakjoller@gmail.com> writes:

> What they suggest is to generate the URL/URI (which is it?)

Every URI identifies some resource.  The URLs, in addition to
identifying resources, also describe where they are located.

So every URL is a URI, but (in theory) not all URIs are URLs.
In practice, though, all the foo://bar things are URLs.

There are also URNs, which just *name* the resources.  For example,
you could have an `isbn' scheme (I don't know if it actually exists),
and the URI <isbn:0-306-40615-2> could be an example of a URN.

An interesting question is whether

   URLs union URNs = URIs?

-- 
Daniel Brockman <daniel@brockman.se>

Re: A few issues with thumbs.el

[Richard Stallman]

    Seems quite good although I don't like the idea of using PNG-files as
    their size is much larger than using JPEG.

    The file names are named according to the absolute URI's MD5 value,
    which most in this thread seems to dislike (including myself).

Would someone like to design a GNU standard for thumbnails?

Re: A few issues with thumbs.el

[Paul Pogonyshev]

Richard Stallman wrote:

>     Seems quite good although I don't like the idea of using PNG-files as
>     their size is much larger than using JPEG.
>
>     The file names are named according to the absolute URI's MD5 value,
>     which most in this thread seems to dislike (including myself).
>
> Would someone like to design a GNU standard for thumbnails?

I would say that GNU standard should be equal to the freedesktop.org standard.
There is absolutely no reason to split over this issue.  If there are problems
with the mentioned proposal, then they should be discussed with its maintaner.

But even if he doesn't amends the draft, why invent a second standard and
start another "standard war"?

Paul

Re: A few issues with thumbs.el

[Richard Stallman]

    >     Seems quite good although I don't like the idea of using PNG-files as
    >     their size is much larger than using JPEG.
    >
    >     The file names are named according to the absolute URI's MD5 value,
    >     which most in this thread seems to dislike (including myself).
    >
    > Would someone like to design a GNU standard for thumbnails?

    I would say that GNU standard should be equal to the freedesktop.org standard.
    There is absolutely no reason to split over this issue.

The "freedesktop.org standard" is simply the preference of a certain
group of people.  The questions they express opinions about are
primarily (though not exclusively) the design conventions of the GNU
system--but they are not part of the GNU Project and do not cooperate
much with the GNU Project.  That is not very nice.

They are entitled to state their preferences, but those preferences
are merely their preferences, nothing more.

If they propose something that is good, we may as well use it.  It
would be silly to deliberately make something inferior.  However,
people have already stated here the reasons why that particular
proposal would be inconvenient.

      If there are problems
    with the mentioned proposal, then they should be discussed with its maintaner.

Whoever wants to talk with them about changing it is welcome to do so.
I will not approach them as a suppliant on behalf of the GNU Project
because that would endorse the unacceptable idea that the GNU Project
needs their approval.  However, it would cause no problem if you talk
with them.

Meanwhile, to prevent things from simply drifting into following their
proposal, I am looking for someone who would like to draw up a GNU
standard for where to store thumbnails.  To have a concrete better
alternative written down cannot hurt the chances of convincing anyone.

Re: A few issues with thumbs.el

[Stefan Monnier]

>> - I wish there was a way to put the thumbs directly next to the original
>> image files.  E.g. instead of ~/.emacs-thumbs/foo!bar!baz!toto!NNNN.jpg
>> I'd like to use ~/foo/bar/baz/.emacs-thumbs/toto_NNNN.jpg.
> What are the benefits of doing this?

Locality.  I find the concept of locality to be extremely important.
E.g. if I rename the directory "foo/bar/baz" to "off/bar/baz" it will just
keep working without any need to recreate thumbs.  If I delete the
directory, the thumbs will automatically be deleted with the images, ...

> I think it would litter the system with extra directories.

What are the problems with that?  It'll still be a negligible fraction of
the disk space and of the number of directories on the disk.

Other advantages: the number of files in a .emacs-thumbs dir will this way
not be larger than the number of files in any other directory, so it'll
automatically avoid suffering from the all-too-common problem where some
Unix file systems don't handle large directories efficiently (things like
O(n) lookup (and worse) behavior).

> I can see how it could solve the uniqueness-problem, not needing to add
> a hash anymore, but I don't know how big a problem that is (see discussion
> below).

I don't care much about solving the uniqueness problem.  I just think that
putting the thumbs next to their source image is The Right Thing.


        Stefan

Re: A few issues with thumbs.el

[Juri Linkov]

>>> - I wish there was a way to put the thumbs directly next to the original
>>> image files.  E.g. instead of ~/.emacs-thumbs/foo!bar!baz!toto!NNNN.jpg
>>> I'd like to use ~/foo/bar/baz/.emacs-thumbs/toto_NNNN.jpg.
>> What are the benefits of doing this?
>
> Locality.  I find the concept of locality to be extremely important.
> E.g. if I rename the directory "foo/bar/baz" to "off/bar/baz" it will just
> keep working without any need to recreate thumbs.  If I delete the
> directory, the thumbs will automatically be deleted with the images, ...

You can't create thumbnails when viewing images in directories
with read-only permissions.

>> I think it would litter the system with extra directories.
>
> What are the problems with that?  It'll still be a negligible fraction of
> the disk space and of the number of directories on the disk.
>
> Other advantages: the number of files in a .emacs-thumbs dir will this way
> not be larger than the number of files in any other directory, so it'll
> automatically avoid suffering from the all-too-common problem where some
> Unix file systems don't handle large directories efficiently (things like
> O(n) lookup (and worse) behavior).

Theoretically this is reasonable, but from a practical POV this approach
is too intrusive.  I had unpleasant experiences with earlier Gimp versions:
often Gimp created thumbnail subdirectories where it really shouldn't do
this, (e.g. in backup directories) when simply previewing image files in
the File Open dialog.  Fortunately Gimp developers realized this annoyance
and fixed it in Gimp 2 to put all thumbnail images in ~/.thumbnails/.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: A few issues with thumbs.el

[Richard Stallman]

    >> - I wish there was a way to put the thumbs directly next to the original
    >> image files.  E.g. instead of ~/.emacs-thumbs/foo!bar!baz!toto!NNNN.jpg
    >> I'd like to use ~/foo/bar/baz/.emacs-thumbs/toto_NNNN.jpg.
    > What are the benefits of doing this?

    Locality.  I find the concept of locality to be extremely important.
    E.g. if I rename the directory "foo/bar/baz" to "off/bar/baz" it will just
    keep working without any need to recreate thumbs.  If I delete the
    directory, the thumbs will automatically be deleted with the images, ...

This would be quite convenient.

Re: A few issues with thumbs.el

[Juanma Barranquero]

Another issue I forgot:

On Windows, thumbs-conversion-program is initialized to "convert.exe".
Trouble is, there's already a convert.exe on the path, on %windir%.

I thought about using (on the windows part) something like

 (let ((exe (executable-find "convert.exe")))
         (and exe
              (string-match "ImageMagick"
                            (shell-command-to-string (concat exe " -version")))
              exe))

Is that appropriate for a defcustom initialization, or it is way too
much work and should be left to the user?

-- 
                    /L/e/k/t/u