No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Morgan Willcock]

When testing the Emacs 30 pre-release I found that all of the PDF files
which I tried to view were not readable because the text appeared to be
missing.

Eventually I found the variable doc-view-mupdf-use-svg, and setting it
to nil reverted the rendering to the old method where the text was
visible.

I suggest that, at the very least, this variable should be added to the
NEWS file so that someone having rendering issues after upgrading can
find it more easily.

I am also not sure that having it default to t (enabling SVG rendering
by default) is a good idea.  It probably works well for certain types of
document but it wasn't usable for the ones which I was viewing (which
were all automated invoices sent by e-mail from multiple organisations).

Thanks,
Morgan

-- 
Morgan Willcock

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Eli Zaretskii]

> From: Morgan Willcock <morgan@ice9.digital>
> Date: Fri, 20 Sep 2024 18:53:44 +0100
> 
> When testing the Emacs 30 pre-release I found that all of the PDF files
> which I tried to view were not readable because the text appeared to be
> missing.
> 
> Eventually I found the variable doc-view-mupdf-use-svg, and setting it
> to nil reverted the rendering to the old method where the text was
> visible.

Why is that?  If your system supports display of SVG images, why
wasn't the text readable?  Tassilo, any ideas?

> I suggest that, at the very least, this variable should be added to the
> NEWS file so that someone having rendering issues after upgrading can
> find it more easily.

Thanks, I added it.

> I am also not sure that having it default to t (enabling SVG rendering
> by default) is a good idea.  It probably works well for certain types of
> document but it wasn't usable for the ones which I was viewing (which
> were all automated invoices sent by e-mail from multiple organisations).

It is only non-nil if your Emacs supports display of SVG images, and
in that case I don't see why not do it.  I think we should investigate
your problem in more detail to understand why it didn't work for you.
I hope Tassilo will have some ideas for looking into this.

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Visuwesh]

[வெள்ளி செப்டம்பர் 20, 2024] Eli Zaretskii wrote:

>> From: Morgan Willcock <morgan@ice9.digital>
>> Date: Fri, 20 Sep 2024 18:53:44 +0100
>> 
>> When testing the Emacs 30 pre-release I found that all of the PDF files
>> which I tried to view were not readable because the text appeared to be
>> missing.
>> 
>> Eventually I found the variable doc-view-mupdf-use-svg, and setting it
>> to nil reverted the rendering to the old method where the text was
>> visible.
>
> Why is that?  If your system supports display of SVG images, why
> wasn't the text readable?  Tassilo, any ideas?
>> I am also not sure that having it default to t (enabling SVG rendering
>> by default) is a good idea.  It probably works well for certain types of
>> document but it wasn't usable for the ones which I was viewing (which
>> were all automated invoices sent by e-mail from multiple organisations).
>
> It is only non-nil if your Emacs supports display of SVG images, and
> in that case I don't see why not do it.  I think we should investigate
> your problem in more detail to understand why it didn't work for you.
> I hope Tassilo will have some ideas for looking into this.

I believe the problem might be the one alluded to in NEWS.29:

    *** doc-view can now generate SVG images when viewing PDF files.
    If Emacs is built with SVG support, doc-view can generate SVG files
    when using MuPDF as the converter for PDF files, which generally leads
    to sharper images (especially when zooming), and allows customization
    of background and foreground color of the page via the new user
    options 'doc-view-svg-background' and 'doc-view-svg-foreground'.  To
    activate this behavior, set 'doc-view-mupdf-use-svg' to non-nil if
    your Emacs has SVG support.  Note that, with some versions of MuPDF,
    SVG generation is known to sometimes produce SVG files that are buggy
    or can take a long time to render.

Stefan reported such a case earlier:
https://yhetil.org/emacs-bugs/jwvbkn722qj.fsf-monnier+emacs@gnu.org/

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Morgan Willcock]

Eli Zaretskii <eliz@gnu.org> writes:

>> Eventually I found the variable doc-view-mupdf-use-svg, and setting it
>> to nil reverted the rendering to the old method where the text was
>> visible.
>
> Why is that?  If your system supports display of SVG images, why
> wasn't the text readable?  Tassilo, any ideas?

I think it is rendering the text and the background of the document in
the same colour.

>> I suggest that, at the very least, this variable should be added to the
>> NEWS file so that someone having rendering issues after upgrading can
>> find it more easily.
>
> Thanks, I added it.

Thanks!

>> I am also not sure that having it default to t (enabling SVG rendering
>> by default) is a good idea.  It probably works well for certain types of
>> document but it wasn't usable for the ones which I was viewing (which
>> were all automated invoices sent by e-mail from multiple organisations).
>
> It is only non-nil if your Emacs supports display of SVG images, and
> in that case I don't see why not do it.  I think we should investigate
> your problem in more detail to understand why it didn't work for you.
> I hope Tassilo will have some ideas for looking into this.

If the problem is down to the default colours not working with
particular documents, then I imagine it will depend on how the PDF files
are being styled.  The documents that I couldn't read included invoices
from Microsoft Azure, so potentially a lot of people could find the same
issue.

-- 
Morgan Willcock

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[chad]

[-- Attachment #1: Type: text/plain, Size: 335 bytes --]

On Fri, Sep 20, 2024 at 3:40 PM Morgan Willcock <morgan@ice9.digital> wrote:

> If the problem is down to the default colours not working with
> particular documents, then I imagine it will depend on how the PDF files
> are being styled.
>

What happens when you try to select/highlight the maybe-fg-equals-bg text?

~Chad

[-- Attachment #2: Type: text/html, Size: 662 bytes --]

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Morgan Willcock]

chad <yandros@gmail.com> writes:

> On Fri, Sep 20, 2024 at 3:40 PM Morgan Willcock <morgan@ice9.digital> wrote:
>
>  If the problem is down to the default colours not working with
>  particular documents, then I imagine it will depend on how the PDF files
>  are being styled.
>
> What happens when you try to select/highlight the maybe-fg-equals-bg text?

If I try to drag a selection with the mouse nothing seems to happen, but
I'm not sure whether that is expected to work for an SVG render.

The other thing of note is that when the document (or at least in the
two I am testing with) is rendered with SVG the number of pages
available seems to increase by 1, but trying to view the final page just
displays this message indefinitely:

  Cannot display this page!
  Maybe because of conversion failure!

-- 
Morgan Willcock

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Eli Zaretskii]

> From: Morgan Willcock <morgan@ice9.digital>
> Cc: Tassilo Horn <tsdh@gnu.org>,   emacs-devel@gnu.org
> Date: Fri, 20 Sep 2024 20:39:47 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Eventually I found the variable doc-view-mupdf-use-svg, and setting it
> >> to nil reverted the rendering to the old method where the text was
> >> visible.
> >
> > Why is that?  If your system supports display of SVG images, why
> > wasn't the text readable?  Tassilo, any ideas?
> 
> I think it is rendering the text and the background of the document in
> the same colour.

In that case, you may wish to customize the new face
doc-view-svg-face.

What do other applications, like a browser, show for the produced SVG
file?  Is it possible that your version of mpdf produces bad SVG
images?

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Morgan Willcock]

Eli Zaretskii <eliz@gnu.org> writes:

>> I think it is rendering the text and the background of the document in
>> the same colour.
>
> In that case, you may wish to customize the new face
> doc-view-svg-face.

It seems to be the currently active theme which determines whether the
the document is readable or not.

If the SVG rendering is enabled (when available) by default, does this
mean that it is the user's responsibility to set usable default colours?
Or should the theme be setting usable default colours?

> What do other applications, like a browser, show for the produced SVG
> file?  Is it possible that your version of mpdf produces bad SVG
> images?

I couldn't find any installed utility by that name, but using image-save
in the DocView buffer suggests that the actual conversion to SVG doesn't
have any problems.  It seems to be purely an issue with how the SVG
image is displayed in the buffer.

-- 
Morgan Willcock

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Tassilo Horn]

Morgan Willcock <morgan@ice9.digital> writes:

>>> I think it is rendering the text and the background of the document
>>> in the same colour.
>>
>> In that case, you may wish to customize the new face
>> doc-view-svg-face.
>
> It seems to be the currently active theme which determines whether the
> the document is readable or not.

The value of the foreground and background doc-view-svg-face properties
are passed as :foreground / :background values to create-image when
inserting the SVG into the buffer.  Their docs are here (info "(elisp)
SVG Images").

--8<---------------cut here---------------start------------->8---
SVG (Scalable Vector Graphics) is an XML format for specifying images.
SVG images support the following additional image descriptor properties:

‘:foreground FOREGROUND’
     FOREGROUND, if non-‘nil’, should be a string specifying a color,
     which is used as the image's foreground color.  If the value is
     ‘nil’, it defaults to the current face's foreground color.

‘:background BACKGROUND’
     BACKGROUND, if non-‘nil’, should be a string specifying a color,
     which is used as the image's background color if the image supports
     transparency.  If the value is ‘nil’, it defaults to the current
     face's background color.
--8<---------------cut here---------------end--------------->8---

Since doc-view-svg-face inherits from the default theme, in theory the
SVG should be adjusted acording to your theme.

> If the SVG rendering is enabled (when available) by default, does this
> mean that it is the user's responsibility to set usable default
> colours?  Or should the theme be setting usable default colours?

The latter.  If the theme sets the default face sensibly (which it
surely does), then your PDFs should look the same.  Well, and that's
true for "normal" PDF documents (black on white, e.g., academic papers)
but somehow fails for "fancy" documents.

I don't know how the :foreground/:background SVG image properties work
but it looks to me as if :background works well every time but
:foreground doesn't as soon as the text is not black.  I suspect the
heuristic is something like: if it's transparent, it's the background,
and if it's either black or white, it's the foreground.

>> What do other applications, like a browser, show for the produced SVG
>> file?  Is it possible that your version of mpdf produces bad SVG
>> images?
>
> I couldn't find any installed utility by that name,

Its mupdf or mudraw.

> but using image-save in the DocView buffer suggests that the actual
> conversion to SVG doesn't have any problems.  It seems to be purely an
> issue with how the SVG image is displayed in the buffer.

You can find the generated SVG files easily using M-x
doc-view-dired-cache RET which opens a dired buffer in the temporary
directory where doc-view stores the files.

Bye,
  Tassilo

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Morgan Willcock]

Tassilo Horn <tsdh@gnu.org> writes:

> Since doc-view-svg-face inherits from the default theme, in theory the
> SVG should be adjusted acording to your theme.

>> If the SVG rendering is enabled (when available) by default, does this
>> mean that it is the user's responsibility to set usable default
>> colours?  Or should the theme be setting usable default colours?
>
> The latter.  If the theme sets the default face sensibly (which it
> surely does), then your PDFs should look the same.  Well, and that's
> true for "normal" PDF documents (black on white, e.g., academic papers)
> but somehow fails for "fancy" documents.
>
> I don't know how the :foreground/:background SVG image properties work
> but it looks to me as if :background works well every time but
> :foreground doesn't as soon as the text is not black.  I suspect the
> heuristic is something like: if it's transparent, it's the background,
> and if it's either black or white, it's the foreground.

When looking at the SVG files in the location browsed by
doc-view-dired-cache I can see that the Azure invoice has:

- A transparent background with black and grey text on it.
- A dark grey border with white text on it.
- A white border with black text on it.

The transparent background is shown as white or black, depending on
whether the light or dark theme is loaded, but the text is always
readable because the text colour complements the background.

The problem is the text that is not on the transparent area of the
document, presumably because a coloured border cannot be identified as
being part of the background and so the border colours will never follow
the theme.

Based on my usage, setting doc-view-mupdf-use-svg to t seems to make
reading PDF documents significantly more difficult.

Perhaps there should be an option to just assume a white background and
render using the original colours, and turn that on by default.  Would
that be what a regular PDF viewer does?

-- 
Morgan Willcock

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Tassilo Horn]

Morgan Willcock <morgan@ice9.digital> writes:

> When looking at the SVG files in the location browsed by
> doc-view-dired-cache I can see that the Azure invoice has:
>
> - A transparent background with black and grey text on it.
> - A dark grey border with white text on it.
> - A white border with black text on it.
>
> The transparent background is shown as white or black, depending on
> whether the light or dark theme is loaded, but the text is always
> readable because the text colour complements the background.
>
> The problem is the text that is not on the transparent area of the
> document, presumably because a coloured border cannot be identified as
> being part of the background and so the border colours will never
> follow the theme.
>
> Based on my usage, setting doc-view-mupdf-use-svg to t seems to make
> reading PDF documents significantly more difficult.
>
> Perhaps there should be an option to just assume a white background
> and render using the original colours, and turn that on by default.

That's what Visuwesh's advice of setting doc-view-svg-face's
foreground/background to black and white would do.  We could do that by
default instead of inheriting from the default face (which is modified
by the theme).  That would probably give sensible results in most cases.

I've also tried un-inheriting doc-view-svg-face from the default face so
that it is the empty face, i.e., all its attributes are nil.  In that
case, it looks exactly as if it inherited from the default face, i.e.,
the document is styled according to the theme with the same problems for
certain files as you see.

I've also created a PDF with blue background and different text colors.
In that case, doc-view-svg-face set to "black on wide" doesn't seem to
have any effect.  The foreground/background colors of the document
appear exactly as I've specified them in the original LaTeX document and
as the Evince or mupdf PDF viewer show them, too.

Eli, what do you think?

> Would that be what a regular PDF viewer does?

It seems so.

Bye,
  Tassilo

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Manuel Giraud via Emacs development discussions.]

Tassilo Horn <tsdh@gnu.org> writes:

[...]

> That's what Visuwesh's advice of setting doc-view-svg-face's
> foreground/background to black and white would do.  We could do that by
> default instead of inheriting from the default face (which is modified
> by the theme).  That would probably give sensible results in most
> cases.

Hi,

When I introduced doc-view-svg-face to inherit from the default face, I
thought it was right for reading long papers to match the user's
preference.  Now, for invoices or other more complex documents, it may
be better to default to "black on white".

But I think we should explain that one could inherit from the default
face to have its theme used into the doc-view-svg-face's docstring.
-- 
Manuel Giraud

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Tassilo Horn]

Manuel Giraud <manuel@ledu-giraud.fr> writes:

Hi Manuel,

> When I introduced doc-view-svg-face to inherit from the default face,
> I thought it was right for reading long papers to match the user's
> preference.

Did you read that when NOT inheriting from the default face and
therefore having all face properties nil also styles according to the
currently active theme?  But yes, you replaced the old variables
doc-view-svg-background/foreground with the new face, and the old
variables where "white"/"black" by default.

> Now, for invoices or other more complex documents, it may be better to
> default to "black on white".

Seems so.

> But I think we should explain that one could inherit from the default
> face to have its theme used into the doc-view-svg-face's docstring.

Certainly.  Would you like to do that?

Thanks,
  Tassilo

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Manuel Giraud via Emacs development discussions.]

Tassilo Horn <tsdh@gnu.org> writes:

> Manuel Giraud <manuel@ledu-giraud.fr> writes:
>
> Hi Manuel,

Hi Tassilo,

>> When I introduced doc-view-svg-face to inherit from the default face,
>> I thought it was right for reading long papers to match the user's
>> preference.
>
> Did you read that when NOT inheriting from the default face and
> therefore having all face properties nil also styles according to the
> currently active theme?  But yes, you replaced the old variables
> doc-view-svg-background/foreground with the new face, and the old
> variables where "white"/"black" by default.

Yes.  It seems that inherinting from default or having all properties to
nil does the same thing.  FWIW, I think we should still use the former
which is more explicit.

>> Now, for invoices or other more complex documents, it may be better to
>> default to "black on white".
>
> Seems so.
>
>> But I think we should explain that one could inherit from the default
>> face to have its theme used into the doc-view-svg-face's docstring.
>
> Certainly.  Would you like to do that?

Yes, I could file a bug report with both changes (face colors and
docstring) but I'd like to hear Eli inputs first.
-- 
Manuel Giraud

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Eli Zaretskii]

> From: Manuel Giraud <manuel@ledu-giraud.fr>
> Cc: Morgan Willcock <morgan@ice9.digital>,  emacs-devel@gnu.org,  Eli
>  Zaretskii <eliz@gnu.org>,  visuweshm@gmail.com
> Date: Mon, 23 Sep 2024 10:34:57 +0200
> 
> When I introduced doc-view-svg-face to inherit from the default face, I
> thought it was right for reading long papers to match the user's
> preference.  Now, for invoices or other more complex documents, it may
> be better to default to "black on white".
> 
> But I think we should explain that one could inherit from the default
> face to have its theme used into the doc-view-svg-face's docstring.

Should we perhaps define suitable settings for this face in the themes
that come with Emacs?

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Tassilo Horn]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Manuel Giraud <manuel@ledu-giraud.fr>
>> Cc: Morgan Willcock <morgan@ice9.digital>,  emacs-devel@gnu.org,  Eli
>>  Zaretskii <eliz@gnu.org>,  visuweshm@gmail.com
>> Date: Mon, 23 Sep 2024 10:34:57 +0200
>> 
>> When I introduced doc-view-svg-face to inherit from the default face, I
>> thought it was right for reading long papers to match the user's
>> preference.  Now, for invoices or other more complex documents, it may
>> be better to default to "black on white".
>> 
>> But I think we should explain that one could inherit from the default
>> face to have its theme used into the doc-view-svg-face's docstring.
>
> Should we perhaps define suitable settings for this face in the themes
> that come with Emacs?

I think the problem is much more likely when using a dark theme, so
those could define black foreground and white background for
doc-view-svg-face.  But since there are so many popular dark themes
outside of emacs, I doubt that's a good idea.

In the end, if the theming of the PDF/SVG gives suitable results depends
on the PDF and we have no way to test if it does for the one at hand.
So I'd rather apply something like that to the emacs-30 branch:

--8<---------------cut here---------------start------------->8---
diff --git a/lisp/doc-view.el b/lisp/doc-view.el
index 15352630d25..667b0e12986 100644
--- a/lisp/doc-view.el
+++ b/lisp/doc-view.el
@@ -238,10 +238,21 @@ doc-view-imenu-flatten
   :type 'boolean
   :version "29.1")
 
-(defface doc-view-svg-face '((t :inherit default))
+(defface doc-view-svg-face '((t :inherit default
+                                :background "white"
+                                :foreground "black"))
   "Face used for SVG images.
 Only background and foreground colors are used.
-See `doc-view-mupdf-use-svg'."
+See `doc-view-mupdf-use-svg'.
+
+If foreground and background color are not specified explicitly and
+therefore inherit from the `default' face, the default
+foreground/background color values are passed to `create-image' when
+preparing the SVG for display in the doc-view buffer.  The effect is
+that the current theme's foreground/background values are applied to the
+SVG display.  Note that this doesn't work for documents which are not
+simply black-on-white where it can lead to unreadable documents due to
+foreground and background color being equal or similar."
   :version "30.1")
 
 (make-obsolete 'doc-view-svg-background 'doc-view-svg-face "30.1")
--8<---------------cut here---------------end--------------->8---

Plus a corresponding NEWS entry telling that unsetting the
foreground/background of doc-view-svg-face will try to apply the current
theme's fg/bg colors also for the display in doc-view (in case of
mupdf+svg).

Bye,
  Tassilo

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Eli Zaretskii]

> From: Tassilo Horn <tsdh@gnu.org>
> Cc: Manuel Giraud <manuel@ledu-giraud.fr>,  morgan@ice9.digital,
>   emacs-devel@gnu.org,  visuweshm@gmail.com
> Date: Mon, 23 Sep 2024 16:51:28 +0200
> 
> > Should we perhaps define suitable settings for this face in the themes
> > that come with Emacs?
> 
> I think the problem is much more likely when using a dark theme, so
> those could define black foreground and white background for
> doc-view-svg-face.  But since there are so many popular dark themes
> outside of emacs, I doubt that's a good idea.

OK.

> In the end, if the theming of the PDF/SVG gives suitable results depends
> on the PDF and we have no way to test if it does for the one at hand.
> So I'd rather apply something like that to the emacs-30 branch:
> 
> --8<---------------cut here---------------start------------->8---
> diff --git a/lisp/doc-view.el b/lisp/doc-view.el
> index 15352630d25..667b0e12986 100644
> --- a/lisp/doc-view.el
> +++ b/lisp/doc-view.el
> @@ -238,10 +238,21 @@ doc-view-imenu-flatten
>    :type 'boolean
>    :version "29.1")
>  
> -(defface doc-view-svg-face '((t :inherit default))
> +(defface doc-view-svg-face '((t :inherit default
> +                                :background "white"
> +                                :foreground "black"))
>    "Face used for SVG images.

This part is okay.

>  Only background and foreground colors are used.
> -See `doc-view-mupdf-use-svg'."
> +See `doc-view-mupdf-use-svg'.
> +
> +If foreground and background color are not specified explicitly and
> +therefore inherit from the `default' face, the default
> +foreground/background color values are passed to `create-image' when
> +preparing the SVG for display in the doc-view buffer.  The effect is
> +that the current theme's foreground/background values are applied to the
> +SVG display.  Note that this doesn't work for documents which are not
> +simply black-on-white where it can lead to unreadable documents due to
> +foreground and background color being equal or similar."
>    :version "30.1")

I think this is too technical, and describes what the face definitions
do.  I think we only need to say there that if the face's colors cause
color SVG images display with low contrast between foreground and
background, the user should customize the face to produce a better
contrast.

Also, I thought we agreed to also have some text in NEWS to delineate
this issue.

Thanks.

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Morgan Willcock]

Eli Zaretskii <eliz@gnu.org> writes:

>> +If foreground and background color are not specified explicitly and
>> +therefore inherit from the `default' face, the default
>> +foreground/background color values are passed to `create-image' when
>> +preparing the SVG for display in the doc-view buffer.  The effect is
>> +that the current theme's foreground/background values are applied to the
>> +SVG display.  Note that this doesn't work for documents which are not
>> +simply black-on-white where it can lead to unreadable documents due to
>> +foreground and background color being equal or similar."
>>    :version "30.1")
>
> I think this is too technical, and describes what the face definitions
> do.  I think we only need to say there that if the face's colors cause
> color SVG images display with low contrast between foreground and
> background, the user should customize the face to produce a better
> contrast.

Unless you know what the document looks like ahead of time, it isn't
obvious that what you are looking at is actually a background and a
foreground.  Perhaps it is best to also mention that elements of the
image may appear to be missing entirely.

-- 
Morgan Willcock

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Tassilo Horn]

Eli Zaretskii <eliz@gnu.org> writes:

>>  Only background and foreground colors are used.
>> -See `doc-view-mupdf-use-svg'."
>> +See `doc-view-mupdf-use-svg'.
>> +
>> +If foreground and background color are not specified explicitly and
>> +therefore inherit from the `default' face, the default
>> +foreground/background color values are passed to `create-image' when
>> +preparing the SVG for display in the doc-view buffer.  The effect is
>> +that the current theme's foreground/background values are applied to the
>> +SVG display.  Note that this doesn't work for documents which are not
>> +simply black-on-white where it can lead to unreadable documents due to
>> +foreground and background color being equal or similar."
>>    :version "30.1")
>
> I think this is too technical, and describes what the face definitions
> do.  I think we only need to say there that if the face's colors cause
> color SVG images display with low contrast between foreground and
> background, the user should customize the face to produce a better
> contrast.

Indeed.

> Also, I thought we agreed to also have some text in NEWS to delineate
> this issue.

Right.  So next try, including a change of the NEWS entry for the face:

--8<---------------cut here---------------start------------->8---
diff --git a/etc/NEWS b/etc/NEWS
index 8ed16cfeeff..c72a87787f3 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1544,10 +1544,12 @@ default is non-nil if your system supports display of SVG images.
 ---
 *** New face 'doc-view-svg-face'.
 This replaces 'doc-view-svg-foreground' and 'doc-view-svg-background'.
-If you don't like the colors produced by the default definition of
-this new face when DocView displays documents, customize this face to
-restore the colors you were used to, or to get colors more to your
-liking.
+By default, this face has black foreground on white background and
+inherits from the default face.  When unsetting the foreground and
+background values, the display in DocView is styled according to the
+current theme.  However, this, or any non-standard values, can result in
+poor contrast for documents which aren't simply black text on white
+background.
 
 ---
 *** DocView buffers now display a new tool bar.
diff --git a/lisp/doc-view.el b/lisp/doc-view.el
index 15352630d25..1ed0c71febe 100644
--- a/lisp/doc-view.el
+++ b/lisp/doc-view.el
@@ -238,10 +238,15 @@ doc-view-imenu-flatten
   :type 'boolean
   :version "29.1")
 
-(defface doc-view-svg-face '((t :inherit default))
+(defface doc-view-svg-face '((t :inherit default
+                                :background "white"
+                                :foreground "black"))
   "Face used for SVG images.
-Only background and foreground colors are used.
-See `doc-view-mupdf-use-svg'."
+See `doc-view-mupdf-use-svg'.
+
+Only background and foreground colors are used as the SVG image's
+descriptors, see (info \"(elisp) SVG Images\").  Non-standard values may
+cause low-contrast issues with certain documents."
   :version "30.1")
 
 (make-obsolete 'doc-view-svg-background 'doc-view-svg-face "30.1")
--8<---------------cut here---------------end--------------->8---

Bye,
  Tassilo

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Eli Zaretskii]

> From: Tassilo Horn <tsdh@gnu.org>
> Cc: manuel@ledu-giraud.fr,  morgan@ice9.digital,  emacs-devel@gnu.org,
>   visuweshm@gmail.com
> Date: Mon, 23 Sep 2024 18:10:34 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I think this is too technical, and describes what the face definitions
> > do.  I think we only need to say there that if the face's colors cause
> > color SVG images display with low contrast between foreground and
> > background, the user should customize the face to produce a better
> > contrast.
> 
> Indeed.
> 
> > Also, I thought we agreed to also have some text in NEWS to delineate
> > this issue.
> 
> Right.  So next try, including a change of the NEWS entry for the face:

LGTM, thanks.

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Tassilo Horn]

Eli Zaretskii <eliz@gnu.org> writes:

>> > Also, I thought we agreed to also have some text in NEWS to
>> > delineate this issue.
>> 
>> Right.  So next try, including a change of the NEWS entry for the face:
>
> LGTM, thanks.

Perfect, pushed to the emacs-30 branch.

Thanks Morgan for making us aware of the issue before the release. :-)

Bye,
  Tassilo

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Eli Zaretskii]

> From: Tassilo Horn <tsdh@gnu.org>
> Cc: manuel@ledu-giraud.fr,  morgan@ice9.digital,  emacs-devel@gnu.org,
>   visuweshm@gmail.com
> Date: Mon, 23 Sep 2024 18:54:16 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> > Also, I thought we agreed to also have some text in NEWS to
> >> > delineate this issue.
> >> 
> >> Right.  So next try, including a change of the NEWS entry for the face:
> >
> > LGTM, thanks.
> 
> Perfect, pushed to the emacs-30 branch.

Did you forget to push?  Because I don't think I see this change on
the branch.

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Tassilo Horn]

Eli Zaretskii <eliz@gnu.org> writes:

>> Perfect, pushed to the emacs-30 branch.
>
> Did you forget to push?  Because I don't think I see this change on
> the branch.

Indeed, I seem to have forgotten.  But now it's really pushed. :-|

Bye,
  Tassilo

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Manuel Giraud via Emacs development discussions.]

Tassilo Horn <tsdh@gnu.org> writes:

[...]

>  
> -(defface doc-view-svg-face '((t :inherit default))
> +(defface doc-view-svg-face '((t :inherit default
> +                                :background "white"
> +                                :foreground "black"))

Since this face is used only for its foreground and background colors, I
think you could leave the :inherit.
-- 
Manuel Giraud

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Tassilo Horn]

Manuel Giraud via "Emacs development discussions." <emacs-devel@gnu.org> writes:

Hi Manuel,

>> -(defface doc-view-svg-face '((t :inherit default))
>> +(defface doc-view-svg-face '((t :inherit default
>> +                                :background "white"
>> +                                :foreground "black"))
>
> Since this face is used only for its foreground and background colors,
> I think you could leave the :inherit.

I know.  But that makes it slightly easier to turn on styling by theme.
Or, well, having both nil without inheriting has the same effect, but as
you have said, explicit is better.  I agree with that, so I kept it on
purpose.

Bye,
  Tassilo

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Manuel Giraud via Emacs development discussions.]

Tassilo Horn <tsdh@gnu.org> writes:

> Manuel Giraud via "Emacs development discussions." <emacs-devel@gnu.org> writes:
>
> Hi Manuel,
>
>>> -(defface doc-view-svg-face '((t :inherit default))
>>> +(defface doc-view-svg-face '((t :inherit default
>>> +                                :background "white"
>>> +                                :foreground "black"))
>>
>> Since this face is used only for its foreground and background colors,
>> I think you could leave the :inherit.
>
> I know.  But that makes it slightly easier to turn on styling by theme.
> Or, well, having both nil without inheriting has the same effect, but as
> you have said, explicit is better.  I agree with that, so I kept it on
> purpose.

Ah ok.  You're right it is easier to configure via custom.  Thanks.
-- 
Manuel Giraud

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Visuwesh]

[திங்கள் செப்டம்பர் 23, 2024] Tassilo Horn wrote:

> Morgan Willcock <morgan@ice9.digital> writes:
>
>> When looking at the SVG files in the location browsed by
>> doc-view-dired-cache I can see that the Azure invoice has:
>>
>> - A transparent background with black and grey text on it.
>> - A dark grey border with white text on it.
>> - A white border with black text on it.
>>
>> The transparent background is shown as white or black, depending on
>> whether the light or dark theme is loaded, but the text is always
>> readable because the text colour complements the background.
>>
>> The problem is the text that is not on the transparent area of the
>> document, presumably because a coloured border cannot be identified as
>> being part of the background and so the border colours will never
>> follow the theme.
>>
>> Based on my usage, setting doc-view-mupdf-use-svg to t seems to make
>> reading PDF documents significantly more difficult.
>>
>> Perhaps there should be an option to just assume a white background
>> and render using the original colours, and turn that on by default.
>
> That's what Visuwesh's advice of setting doc-view-svg-face's
> foreground/background to black and white would do.  We could do that by
> default instead of inheriting from the default face (which is modified
> by the theme).  That would probably give sensible results in most cases.
>
> I've also tried un-inheriting doc-view-svg-face from the default face so
> that it is the empty face, i.e., all its attributes are nil.  In that
> case, it looks exactly as if it inherited from the default face, i.e.,
> the document is styled according to the theme with the same problems for
> certain files as you see.
>
> I've also created a PDF with blue background and different text colors.
> In that case, doc-view-svg-face set to "black on wide" doesn't seem to
> have any effect.  The foreground/background colors of the document
> appear exactly as I've specified them in the original LaTeX document and
> as the Evince or mupdf PDF viewer show them, too.
>
> Eli, what do you think?
>
>> Would that be what a regular PDF viewer does?
>
> It seems so.

AFAIU, the "dark-mode" PDF viewers offer has a lot that goes into it
especially wrt making images actually viewable.  Since we don't do any
such complex handling (nor do we instruct mutool to do it), we present
an unreadable document in this case.  I believe it would be best to
default the face to be black-on-white, and let the user customise the
face to match whatever colour they want at their own risk.

> Bye,
>   Tassilo

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Eli Zaretskii]

> From: Tassilo Horn <tsdh@gnu.org>
> Cc: emacs-devel@gnu.org,  Eli Zaretskii <eliz@gnu.org>, visuweshm@gmail.com
> Date: Mon, 23 Sep 2024 08:44:35 +0200
> 
> Morgan Willcock <morgan@ice9.digital> writes:
> 
> > When looking at the SVG files in the location browsed by
> > doc-view-dired-cache I can see that the Azure invoice has:
> >
> > - A transparent background with black and grey text on it.
> > - A dark grey border with white text on it.
> > - A white border with black text on it.
> >
> > The transparent background is shown as white or black, depending on
> > whether the light or dark theme is loaded, but the text is always
> > readable because the text colour complements the background.
> >
> > The problem is the text that is not on the transparent area of the
> > document, presumably because a coloured border cannot be identified as
> > being part of the background and so the border colours will never
> > follow the theme.
> >
> > Based on my usage, setting doc-view-mupdf-use-svg to t seems to make
> > reading PDF documents significantly more difficult.
> >
> > Perhaps there should be an option to just assume a white background
> > and render using the original colours, and turn that on by default.
> 
> That's what Visuwesh's advice of setting doc-view-svg-face's
> foreground/background to black and white would do.  We could do that by
> default instead of inheriting from the default face (which is modified
> by the theme).  That would probably give sensible results in most cases.
> 
> I've also tried un-inheriting doc-view-svg-face from the default face so
> that it is the empty face, i.e., all its attributes are nil.  In that
> case, it looks exactly as if it inherited from the default face, i.e.,
> the document is styled according to the theme with the same problems for
> certain files as you see.
> 
> I've also created a PDF with blue background and different text colors.
> In that case, doc-view-svg-face set to "black on wide" doesn't seem to
> have any effect.  The foreground/background colors of the document
> appear exactly as I've specified them in the original LaTeX document and
> as the Evince or mupdf PDF viewer show them, too.
> 
> Eli, what do you think?

Sorry, I don't understand what you are asking me about.  Please
elaborate

It sounds like PDF documents with colors have inherent problems with
doc-view-mupdf-use-svg and/or themes, so maybe we should warn about
this in NEWS and in the doc string, and suggest that people either
customize the doc-view-svg-face, or disable doc-view-mupdf-use-svg.

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Tassilo Horn]

Eli Zaretskii <eliz@gnu.org> writes:

>> Eli, what do you think?
>
> Sorry, I don't understand what you are asking me about.  Please
> elaborate
>
> It sounds like PDF documents with colors have inherent problems with
> doc-view-mupdf-use-svg and/or themes, so maybe we should warn about
> this in NEWS and in the doc string, and suggest that people either
> customize the doc-view-svg-face, or disable doc-view-mupdf-use-svg.

Yes, exactly.  There can be foreground-similar-to-background color
issues when mupdf + doc-view-mupdf-use-svg is used for certain, not
uncommon documents.

We can either document that issue in NEWS and docstrings, or we can
simply set "black on white" as default value of doc-view-svg-face which
would eliminate those problems (and match what we had in emacs 29 before
the face was introduced by Manuel Giraud) but would stop trying to apply
our emacs theme to the document, too.

I'd rather be conservative, use "black on white" as doc-view-svg-face
default value and document in NEWS and doc-view-svg-face's docstring
that one can style documents according to the emacs theme by inheriting
from the default face and not explicitly setting foreground and
background colors.

Bye,
  Tassilo

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Eli Zaretskii]

> From: Tassilo Horn <tsdh@gnu.org>
> Cc: morgan@ice9.digital,  emacs-devel@gnu.org,  visuweshm@gmail.com, Manuel
>  Giraud <manuel@ledu-giraud.fr>
> Date: Mon, 23 Sep 2024 14:05:30 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Eli, what do you think?
> >
> > Sorry, I don't understand what you are asking me about.  Please
> > elaborate
> >
> > It sounds like PDF documents with colors have inherent problems with
> > doc-view-mupdf-use-svg and/or themes, so maybe we should warn about
> > this in NEWS and in the doc string, and suggest that people either
> > customize the doc-view-svg-face, or disable doc-view-mupdf-use-svg.
> 
> Yes, exactly.  There can be foreground-similar-to-background color
> issues when mupdf + doc-view-mupdf-use-svg is used for certain, not
> uncommon documents.
> 
> We can either document that issue in NEWS and docstrings, or we can
> simply set "black on white" as default value of doc-view-svg-face which
> would eliminate those problems (and match what we had in emacs 29 before
> the face was introduced by Manuel Giraud) but would stop trying to apply
> our emacs theme to the document, too.
> 
> I'd rather be conservative, use "black on white" as doc-view-svg-face
> default value and document in NEWS and doc-view-svg-face's docstring
> that one can style documents according to the emacs theme by inheriting
> from the default face and not explicitly setting foreground and
> background colors.

Agreed.

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Tassilo Horn]

Eli Zaretskii <eliz@gnu.org> writes:

>> When testing the Emacs 30 pre-release I found that all of the PDF
>> files which I tried to view were not readable because the text
>> appeared to be missing.
>> 
>> Eventually I found the variable doc-view-mupdf-use-svg, and setting
>> it to nil reverted the rendering to the old method where the text was
>> visible.
>
> Why is that?  If your system supports display of SVG images, why
> wasn't the text readable?  Tassilo, any ideas?

I suspect it's some sort of background-equals-foreground issue.  I think
by default, the SVGs have transparent background.  So when you use a
dark theme (I tested with the built-in modus-vivendi theme which is very
dark), you might get black foreground on black background.

Visuwesh already cited the cure: in emacs 29 you might want to set
doc-view-svg-foreground/background to white/black or vice versa (and
then re-convert your PDF).

In emacs 30, that has been replaced with the face doc-view-svg-face
where you can customize foreground and background such that the contrast
works for you and your theme.

Hm, but doc-view-svg-face inherits from the default face and that is
white-on-black with modus-vivendi, so actually it should all be good,
i.e., the PDF display should match your theme.  However, depending on
PDF I tested, I sometimes get white-on-black display (good!) and
sometimes darkish-on-black (bad!).  It seems like it works as intended
for "standard black-on-white PDFs" but doesn't really work when the PDF
is "fancy", e.g., gray foreground or colorized borders or what not...

Bye,
  Tassilo

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Morgan Willcock]

Tassilo Horn <tsdh@gnu.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> When testing the Emacs 30 pre-release I found that all of the PDF
>>> files which I tried to view were not readable because the text
>>> appeared to be missing.
>>>
>>> Eventually I found the variable doc-view-mupdf-use-svg, and setting
>>> it to nil reverted the rendering to the old method where the text was
>>> visible.
>>
>> Why is that?  If your system supports display of SVG images, why
>> wasn't the text readable?  Tassilo, any ideas?
>
> I suspect it's some sort of background-equals-foreground issue.  I think
> by default, the SVGs have transparent background.  So when you use a
> dark theme (I tested with the built-in modus-vivendi theme which is very
> dark), you might get black foreground on black background.

I use modus-vivendi and sometimes toggle it to modus-operandi to use the
light version.  It does seem to be the currently active theme which
makes the text invisible, with the dark theme making the text
unreadable.

If I load no theme then the documents are rendered normally (as far as I
can tell).

> Visuwesh already cited the cure: in emacs 29 you might want to set
> doc-view-svg-foreground/background to white/black or vice versa (and
> then re-convert your PDF).

If I use image-save to save a copy of the SVG file, the conversion
always appears to be usable and a good representation of the original
document, even when what is displayed in the buffer is not.

Toggling the theme between the light and dark theme while viewing the
image in the buffer toggles it between readable and unreadable.

> In emacs 30, that has been replaced with the face doc-view-svg-face
> where you can customize foreground and background such that the contrast
> works for you and your theme.
>
> Hm, but doc-view-svg-face inherits from the default face and that is
> white-on-black with modus-vivendi, so actually it should all be good,
> i.e., the PDF display should match your theme.  However, depending on
> PDF I tested, I sometimes get white-on-black display (good!) and
> sometimes darkish-on-black (bad!).

Is there any way to opt-out from the theme having any effect on how the
result is presented while still giving a usable result?

> It seems like it works as intended for "standard black-on-white PDFs"
> but doesn't really work when the PDF is "fancy", e.g., gray foreground
> or colorized borders or what not...

This does describe the documents I am testing with.  They all have some
kind of coloured border.  Azure invoices use a light gray foreground
colour with black text on it, while also using a darker gray background
with white text on it.

-- 
Morgan Willcock

Re: No NEWS entry for doc-view-mupdf-use-svg (Emacs 30.0.91 feedback)

[Visuwesh]

[சனி செப்டம்பர் 21, 2024] Morgan Willcock wrote:

> Tassilo Horn <tsdh@gnu.org> writes:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>>> When testing the Emacs 30 pre-release I found that all of the PDF
>>>> files which I tried to view were not readable because the text
>>>> appeared to be missing.
>>>>
>>>> Eventually I found the variable doc-view-mupdf-use-svg, and setting
>>>> it to nil reverted the rendering to the old method where the text was
>>>> visible.
>>>
>>> Why is that?  If your system supports display of SVG images, why
>>> wasn't the text readable?  Tassilo, any ideas?
>>
>> I suspect it's some sort of background-equals-foreground issue.  I think
>> by default, the SVGs have transparent background.  So when you use a
>> dark theme (I tested with the built-in modus-vivendi theme which is very
>> dark), you might get black foreground on black background.
>
> I use modus-vivendi and sometimes toggle it to modus-operandi to use the
> light version.  It does seem to be the currently active theme which
> makes the text invisible, with the dark theme making the text
> unreadable.
>
> If I load no theme then the documents are rendered normally (as far as I
> can tell).
>
>> Visuwesh already cited the cure: in emacs 29 you might want to set
>> doc-view-svg-foreground/background to white/black or vice versa (and
>> then re-convert your PDF).
>
> If I use image-save to save a copy of the SVG file, the conversion
> always appears to be usable and a good representation of the original
> document, even when what is displayed in the buffer is not.
>
> Toggling the theme between the light and dark theme while viewing the
> image in the buffer toggles it between readable and unreadable.
>
>> In emacs 30, that has been replaced with the face doc-view-svg-face
>> where you can customize foreground and background such that the contrast
>> works for you and your theme.
>>
>> Hm, but doc-view-svg-face inherits from the default face and that is
>> white-on-black with modus-vivendi, so actually it should all be good,
>> i.e., the PDF display should match your theme.  However, depending on
>> PDF I tested, I sometimes get white-on-black display (good!) and
>> sometimes darkish-on-black (bad!).
>
> Is there any way to opt-out from the theme having any effect on how the
> result is presented while still giving a usable result?

You can customise the face doc-view-svg-face to have :background "white"
and :foreground "black".

>> It seems like it works as intended for "standard black-on-white PDFs"
>> but doesn't really work when the PDF is "fancy", e.g., gray foreground
>> or colorized borders or what not...
>
> This does describe the documents I am testing with.  They all have some
> kind of coloured border.  Azure invoices use a light gray foreground
> colour with black text on it, while also using a darker gray background
> with white text on it.