bug#65188: 29.1; WEBP image not displayed

bug#65188: 29.1; WEBP image not displayed

[awrhygty]

I installed emacs 29 from:
  https://ftp.gnu.org/gnu/emacs/windows/emacs-29/emacs-29.1_1-installer.exe
When find-file with WEBP images, they are not displayed visually,
and emacs reports:
  Cannot display image: (Invalid image specification)
There exists c:/Emacs/emacs-29.1/bin/libwebp-7.dll in the same directory
 of emacs.exe.

image-types
=> (bmp svg webp png gif tiff jpeg xpm xbm pbm)
(image-type-available-p 'webp)
=> nil
dynamic-library-alist
=> ((gdiplus "gdiplus.dll")
    (shlwapi "shlwapi.dll")
    (xpm "libxpm.dll" "xpm4.dll" "libXpm-nox4.dll")
    (png "libpng16.dll" "libpng16-16.dll")
    (tiff "libtiff-5.dll" "libtiff3.dll" "libtiff.dll")
    (jpeg "libjpeg-8.dll")
    (gif "libgif-7.dll")
    (svg "librsvg-2-2.dll")
    (webp "libwebp-7.dll" "libwebp.dll")
    (webpdemux "libwebpdemux-2.dll" "libwebpdemux.dll")
    (sqlite3 "libsqlite3-0.dll")
    (gdk-pixbuf "libgdk_pixbuf-2.0-0.dll")
    (glib "libglib-2.0-0.dll")
    (gio "libgio-2.0-0.dll")
    (gobject "libgobject-2.0-0.dll")
    (gnutls "libgnutls-30.dll")
    (libxml2 "libxml2-2.dll" "libxml2.dll")
    (zlib "zlib1.dll" "libz-1.dll")
    (lcms2 "liblcms2-2.dll")
    (json "libjansson-4.dll")
    (gccjit "libgccjit-0.dll")
    (tree-sitter "libtree-sitter.dll" "libtree-sitter-0.dll"))



In GNU Emacs 29.1 (build 2, x86_64-w64-mingw32) of 2023-08-02 built on
 AVALON
Windowing system distributor 'Microsoft Corp.', version 10.0.19045
System Description: Microsoft Windows 10 Pro (v10.0.2009.19045.3324)

Configured using:
 'configure --with-modules --without-dbus --with-native-compilation=aot
 --without-compress-install --with-tree-sitter CFLAGS=-O2'

Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY W32NOTIFY PDUMPER PNG RSVG SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XPM ZLIB

(NATIVE_COMP present but libgccjit not available)

Important settings:
  value of $LANG: JPN
  locale-coding-system: cp932

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(gnutls network-stream nsm mailalias smtpmail textsec uni-scripts url
url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util url-parse auth-source cl-seq
eieio eieio-core cl-macs json map byte-opt gv bytecomp byte-compile
url-vars idna-mapping ucs-normalize uni-confusable textsec-check
misearch multi-isearch help-mode pp shadow sort mail-extr emacsbug
message mailcap yank-media puny rfc822 mml mml-sec password-cache epa
derived epg rfc6068 epg-config gnus-util text-property-search time-date
subr-x mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils image-mode dired dired-loaddefs exif find-func
thingatpt cl-loaddefs cl-lib term/bobcat japan-util rmc iso-transl
tooltip cconv eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel dos-w32 ls-lisp disp-table
term/w32-win w32-win w32-vars term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads w32notify w32 lcms2 multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 219080 14406)
 (symbols 48 7582 2)
 (strings 32 37522 1848)
 (string-bytes 1 1037683)
 (vectors 16 39585)
 (vector-slots 8 796659 39426)
 (floats 8 62 200)
 (intervals 56 832 8)
 (buffers 984 14))

bug#65188: 29.1; WEBP image not displayed

[Eli Zaretskii]

> From: awrhygty@outlook.com
> Date: Thu, 10 Aug 2023 09:20:52 +0900
> 
> 
> I installed emacs 29 from:
>   https://ftp.gnu.org/gnu/emacs/windows/emacs-29/emacs-29.1_1-installer.exe
> When find-file with WEBP images, they are not displayed visually,
> and emacs reports:
>   Cannot display image: (Invalid image specification)
> There exists c:/Emacs/emacs-29.1/bin/libwebp-7.dll in the same directory
>  of emacs.exe.
> 
> image-types
> => (bmp svg webp png gif tiff jpeg xpm xbm pbm)
> (image-type-available-p 'webp)
> => nil

This might mean your libwebp-7.dll is incompatible with Emacs you have
(e.g., it's a 32-bit DLL, not 64-bit DLL), or Emacs cannot load it for
some other reason.

Corwin, can you please verify that the binary on the GNU FTP site can
display WebP images when a suitable libwebp DLL is available?  The
Emacs 29.1 I built myself here is definitely capable of displaying
WebP images.

bug#65188: 29.1; WEBP image not displayed

[Corwin Brust]

On Thu, Aug 10, 2023 at 1:34 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: awrhygty@outlook.com
> > Date: Thu, 10 Aug 2023 09:20:52 +0900
> >
> >
> > I installed emacs 29 from:
> >   https://ftp.gnu.org/gnu/emacs/windows/emacs-29/emacs-29.1_1-installer.exe
> > When find-file with WEBP images, they are not displayed visually,
> > and emacs reports:
> >   Cannot display image: (Invalid image specification)
> > There exists c:/Emacs/emacs-29.1/bin/libwebp-7.dll in the same directory
> >  of emacs.exe.
> >
> > image-types
> > => (bmp svg webp png gif tiff jpeg xpm xbm pbm)
> > (image-type-available-p 'webp)
> > => nil
>
> This might mean your libwebp-7.dll is incompatible with Emacs you have
> (e.g., it's a 32-bit DLL, not 64-bit DLL), or Emacs cannot load it for
> some other reason.
>
> Corwin, can you please verify that the binary on the GNU FTP site can
> display WebP images when a suitable libwebp DLL is available?  The
> Emacs 29.1 I built myself here is definitely capable of displaying
> WebP images.
>

Yes, AFAICT it is working as expected. I used ProcessExplorer
(filtering on Emacs then Search | DLL for the webp) to check Emacs
really used the DLL  from the full-zip.  I did this from
emacs-29.1_1.zip from GNU FTP (after first doing the same steps using
my local "cache" of the last release  build's PREFIX folder, keep
around for this sort of question).

Predictably:
(image-type-available 'webp) => t, over here

bug#65188: 29.1; WEBP image not displayed

[Eli Zaretskii]

> From: Corwin Brust <corwin@bru.st>
> Date: Thu, 10 Aug 2023 02:24:38 -0500
> Cc: awrhygty@outlook.com, 65188@debbugs.gnu.org
> 
> On Thu, Aug 10, 2023 at 1:34 AM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > > From: awrhygty@outlook.com
> > > Date: Thu, 10 Aug 2023 09:20:52 +0900
> > >
> > >
> > > I installed emacs 29 from:
> > >   https://ftp.gnu.org/gnu/emacs/windows/emacs-29/emacs-29.1_1-installer.exe
> > > When find-file with WEBP images, they are not displayed visually,
> > > and emacs reports:
> > >   Cannot display image: (Invalid image specification)
> > > There exists c:/Emacs/emacs-29.1/bin/libwebp-7.dll in the same directory
> > >  of emacs.exe.
> > >
> > > image-types
> > > => (bmp svg webp png gif tiff jpeg xpm xbm pbm)
> > > (image-type-available-p 'webp)
> > > => nil
> >
> > This might mean your libwebp-7.dll is incompatible with Emacs you have
> > (e.g., it's a 32-bit DLL, not 64-bit DLL), or Emacs cannot load it for
> > some other reason.
> >
> > Corwin, can you please verify that the binary on the GNU FTP site can
> > display WebP images when a suitable libwebp DLL is available?  The
> > Emacs 29.1 I built myself here is definitely capable of displaying
> > WebP images.
> >
> 
> Yes, AFAICT it is working as expected. I used ProcessExplorer
> (filtering on Emacs then Search | DLL for the webp) to check Emacs
> really used the DLL  from the full-zip.  I did this from
> emacs-29.1_1.zip from GNU FTP (after first doing the same steps using
> my local "cache" of the last release  build's PREFIX folder, keep
> around for this sort of question).
> 
> Predictably:
> (image-type-available 'webp) => t, over here

Thanks.  So this leaves us with the other possibility: that
libwebp-7.dll on the OP's machine is the cause of the problem.  Where
did you download that DLL?

bug#65188: 29.1; WEBP image not displayed

[Corwin Brust]

On Thu, Aug 10, 2023 at 2:45 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Corwin Brust <corwin@bru.st>
> > Date: Thu, 10 Aug 2023 02:24:38 -0500
> > Cc: awrhygty@outlook.com, 65188@debbugs.gnu.org
> >
> > On Thu, Aug 10, 2023 at 1:34 AM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > Yes, AFAICT it is working as expected. I used ProcessExplorer

Hang on a minute...

I played with ProcessExplorer a little bit more, this time exporting
the list of DLLs for emacs-29.1.exe instead of the search/filter
approach looking specifically for libwebp-7.  Here I can see two webp
related DLLs (but I'm fairly sure we only include one of them).  The
one I didn't expect to find (libwebpdemux) is found from install of
MSYS, and seems to be about container parsing.

libwebpdemux-2.dll C:\msys2\mingw64\bin\libwebpdemux-2.dll
libwebp-7.dll H:\packaging-test\emacs-29.1_1\bin\libwebp-7.dll

Problem, yes?   Packaging problem only?

I'm sorry, I wasn't aware I needed this.  I'm happy to start on
updating the deps/deps-sources and making a new set of binaries "this
should have been included" is the correct surmise.

bug#65188: 29.1; WEBP image not displayed

[Corwin Brust]

On Thu, Aug 10, 2023 at 3:04 AM Corwin Brust <corwin@bru.st> wrote:
>
> approach looking specifically for libwebp-7.  Here I can see two webp
> related DLLs (but I'm fairly sure we only include one of them).  The
>
> libwebpdemux-2.dll C:\msys2\mingw64\bin\libwebpdemux-2.dll
> libwebp-7.dll H:\packaging-test\emacs-29.1_1\bin\libwebp-7.dll
>
> Problem, yes?   Packaging problem only?

User jbe on Libera.Chat #emacs IRC kindly confirmed both the original
issue reproduces using the emacs-29.1_1 binaries, and that adding the
"debux" DLL mentioned above resolves the issue.

So, I have created a new deps archive and am working on a new release
set.  Once I have created and verified a new set of binaries, if there
are not concerns or countermanding instructions, I'll go ahead and
push emacs-29.1_2 to GNU FTP.

1. New deps zip with more webp DLLs:
https://corwin.bru.st/emacs-29/emacs-29-2023-08-10-deps.zip

2. In the same folder are dependency sources (src-deps) and sig files:
https://corwin.bru.st/emacs-29/


awrhygty (and anyone),

You can test that the new set will fix the problem for you by
unpacking the deps zip file (link #1, above) and unpacking it on atop
the bin folder from your emacs-29.1_1 install.  It doesn't matter
whether you say "Yes" to allow replacing files or not - the only
differences should be some new WEBP related DLLs.

Sorry for the inconvenience and thanks for reporting the problem.

Eli,

I have expressly added WEBP into the list of packages needed, in
admin/nt/build-deps-zip.py.   It is worth noting that this script is
really painful because the upstream MSYS package source paths keep
changing; I had to make several unrelated (cert and sqllite3 source
archive URL) changes to run it, just now.   I think lib-webp7.dll was
included at all only because something else we list as a dep was
depending on it, but I'm not quite sure what just yet.

I realize others don't find this script as important as I do, but I've
really struggled to do what it does manually constantly, much less
create simpler scripting.  I regret not sending in a proper patch for
discussion well in advance of the release.

Is this the right place, or should I open my own bug report to discuss
my version of build-deps-zips and whether to merge my changes that
make it work for building Emacs 29 and Enacs 30?  (Not attaching
anything quite yet, but I did put the current version of the script
into the folder I linked above.)

bug#65188: 29.1; WEBP image not displayed

[Eli Zaretskii]

> From: Corwin Brust <corwin@bru.st>
> Date: Thu, 10 Aug 2023 04:16:08 -0500
> Cc: awrhygty@outlook.com, 65188@debbugs.gnu.org
> 
> I have expressly added WEBP into the list of packages needed, in
> admin/nt/build-deps-zip.py.   It is worth noting that this script is
> really painful because the upstream MSYS package source paths keep
> changing; I had to make several unrelated (cert and sqllite3 source
> archive URL) changes to run it, just now.   I think lib-webp7.dll was
> included at all only because something else we list as a dep was
> depending on it, but I'm not quite sure what just yet.
> 
> I realize others don't find this script as important as I do, but I've
> really struggled to do what it does manually constantly, much less
> create simpler scripting.  I regret not sending in a proper patch for
> discussion well in advance of the release.
> 
> Is this the right place, or should I open my own bug report to discuss
> my version of build-deps-zips and whether to merge my changes that
> make it work for building Emacs 29 and Enacs 30?  (Not attaching
> anything quite yet, but I did put the current version of the script
> into the folder I linked above.)

I think this is worth a separate bug report, yes.  I understand that
playing catch with MSYS distros is a pain, and agree that we should
find a better way if it exists.  But let's discuss it separately; this
bug and its remediation are clear.

bug#65188: 29.1; WEBP image not displayed

[Corwin Brust]

On Thu, Aug 10, 2023 at 4:16 AM Corwin Brust <corwin@bru.st> wrote:
>
> On Thu, Aug 10, 2023 at 3:04 AM Corwin Brust <corwin@bru.st> wrote:
> >
> > approach looking specifically for libwebp-7.  Here I can see two webp
> > related DLLs (but I'm fairly sure we only include one of them).  The
> >
> > libwebpdemux-2.dll C:\msys2\mingw64\bin\libwebpdemux-2.dll
> > libwebp-7.dll H:\packaging-test\emacs-29.1_1\bin\libwebp-7.dll
> >
> > Problem, yes?   Packaging problem only?
>
> User jbe on Libera.Chat #emacs IRC kindly confirmed both the original
> issue reproduces using the emacs-29.1_1 binaries, and that adding the
> "debux" DLL mentioned above resolves the issue.
>
> So, I have created a new deps archive and am working on a new release
> set.  Once I have created and verified a new set of binaries, if there
> are not concerns or countermanding instructions, I'll go ahead and
> push emacs-29.1_2 to GNU FTP.
>

>
> awrhygty (and anyone),
>
> You can test that the new set will fix the problem for you by

either
 - running emacs-29.1_2-installer.exe, or else
 - unpacking emacs-29.1_2.zip,
and then trying to load a WEBP image on the resulting Emacs

> unpacking the deps zip file (link #1, above) and unpacking it on atop
> the bin folder from your emacs-29.1_1 install.  It doesn't matter
> whether you say "Yes" to allow replacing files or not - the only
> differences should be some new WEBP related DLLs.
>
> Sorry for the inconvenience and thanks for reporting the problem.
>
> Eli,
>
> I have expressly added WEBP into the list of packages needed, in
> admin/nt/build-deps-zip.py.   It is worth noting that this script is
> really painful because the upstream MSYS package source paths keep
> changing; I had to make several unrelated (cert and sqllite3 source
> archive URL) changes to run it, just now.   I think lib-webp7.dll was
> included at all only because something else we list as a dep was
> depending on it, but I'm not quite sure what just yet.
>
> I realize others don't find this script as important as I do, but I've
> really struggled to do what it does manually constantly, much less
> create simpler scripting.  I regret not sending in a proper patch for
> discussion well in advance of the release.
>
> Is this the right place, or should I open my own bug report to discuss
> my version of build-deps-zips and whether to merge my changes that
> make it work for building Emacs 29 and Enacs 30?  (Not attaching
> anything quite yet, but I did put the current version of the script
> into the folder I linked above.)

Now done.  You should soon find emacs-29.1_2 on your favorite GNU FTP
mirror.  Meanwhile, here's a non-mirror link in case you are seeing
this (as I am writing it), before propagation to mirrors:

  https://ftp.gnu.org/gnu/emacs/windows/emacs-29/?C=M;O=D

awrhygty,

Please try emacs-29.1_2.zip or (emacs-29.1_2-installer.exe) and
confirm this version renders WEBP images for you.

TIA and (again) for reporting

bug#65188: 29.1; WEBP image not displayed

[Corwin Brust]

CEZB: Ug, please ignore my last.  Terrible mess of multiple drafts in there.

awrhygty,

New (I hope, corrected) binaries are uploaded as emacs-29.1_2.

  https://ftp.gnu.org/gnu/emacs/windows/emacs-29/?C=M;O=D

Please try emacs-29.1_2.zip or (emacs-29.1_2-installer.exe) and
confirm this version renders WEBP images for you.

bug#65188: 29.1; WEBP image not displayed

[awrhygty]

Corwin Brust <corwin@bru.st> writes:
> CEZB: Ug, please ignore my last.  Terrible mess of multiple drafts in there.
>
> awrhygty,
>
> New (I hope, corrected) binaries are uploaded as emacs-29.1_2.
>
>   https://ftp.gnu.org/gnu/emacs/windows/emacs-29/?C=M;O=D
>
> Please try emacs-29.1_2.zip or (emacs-29.1_2-installer.exe) and
> confirm this version renders WEBP images for you.

I tried the new installer:
  https://ftp.gnu.org/gnu/emacs/windows/emacs-29/emacs-29.1_2-installer.exe
Now I can view WEBP images visually.
Thank you.

bug#65188: 29.1; WEBP image not displayed

[Eli Zaretskii]

> From: awrhygty@outlook.com
> Cc: 65188@debbugs.gnu.org,  Eli Zaretskii <eliz@gnu.org>
> Date: Thu, 10 Aug 2023 22:39:37 +0900
> 
> Corwin Brust <corwin@bru.st> writes:
> > CEZB: Ug, please ignore my last.  Terrible mess of multiple drafts in there.
> >
> > awrhygty,
> >
> > New (I hope, corrected) binaries are uploaded as emacs-29.1_2.
> >
> >   https://ftp.gnu.org/gnu/emacs/windows/emacs-29/?C=M;O=D
> >
> > Please try emacs-29.1_2.zip or (emacs-29.1_2-installer.exe) and
> > confirm this version renders WEBP images for you.
> 
> I tried the new installer:
>   https://ftp.gnu.org/gnu/emacs/windows/emacs-29/emacs-29.1_2-installer.exe
> Now I can view WEBP images visually.

Great, thanks for testing.  I'm therefore closing this bug.

bug#65188: 29.1; WEBP image not displayed

[Eli Zaretskii]

> From: Corwin Brust <corwin@bru.st>
> Date: Thu, 10 Aug 2023 03:04:45 -0500
> Cc: awrhygty@outlook.com, 65188@debbugs.gnu.org
> 
> I played with ProcessExplorer a little bit more, this time exporting
> the list of DLLs for emacs-29.1.exe instead of the search/filter
> approach looking specifically for libwebp-7.  Here I can see two webp
> related DLLs (but I'm fairly sure we only include one of them).  The
> one I didn't expect to find (libwebpdemux) is found from install of
> MSYS, and seems to be about container parsing.
> 
> libwebpdemux-2.dll C:\msys2\mingw64\bin\libwebpdemux-2.dll
> libwebp-7.dll H:\packaging-test\emacs-29.1_1\bin\libwebp-7.dll
> 
> Problem, yes?   Packaging problem only?

You are saying that the "dependencies" zip lacks libwebpdemux-2.dll?
If so, then yes, it's a packaging problem, since Emacs needs to load
that DLL to support WebP images.

> I'm sorry, I wasn't aware I needed this.  I'm happy to start on
> updating the deps/deps-sources and making a new set of binaries "this
> should have been included" is the correct surmise.

OK, TIA.