[PATCH] ffap.el: Exclude angle brackets from file names in XML

[PATCH] ffap.el: Exclude angle brackets from file names in XML

[StrawberryTea]

In my company's code base, we have a lot of XML files that contain file
names in tags.  For example:

  <file>foo/bar.txt</file>

Currently, ffap.el does not recognize these file names as such, because
it recognizes the angle brackets as part of the file name.  This patch
fixes that.
---
 lisp/ffap.el | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lisp/ffap.el b/lisp/ffap.el
index 5383f743878..b2b681b7c44 100644
--- a/lisp/ffap.el
+++ b/lisp/ffap.el
@@ -1065,6 +1065,9 @@ ffap-string-at-point-mode-alist
     ;; (La)TeX: don't allow braces
     (latex-mode "--:\\\\$+<>@-Z_[:alpha:]~*?" "<@" "@>;.,!:")
     (tex-mode "--:\\\\$+<>@-Z_[:alpha:]~*?" "<@" "@>;.,!:")
+    ;; XML: don't allow angle brackets
+    (xml-mode "--:\\\\${}+@-Z_[:alpha:]~*?#" "{<@" "@>;.,!:}")
+    (nxml-mode "--:\\\\${}+@-Z_[:alpha:]~*?#" "{<@" "@>;.,!:}")
     )
   "Alist of (MODE CHARS BEG END), where MODE is a symbol.
 This is possibly a major-mode name, or one of the symbols
--
2.44.0

--
StrawberryTea

Re: [PATCH] ffap.el: Exclude angle brackets from file names in XML

[Eli Zaretskii]

> From: StrawberryTea <look@strawberrytea.xyz>
> Date: Sat, 9 Mar 2024 15:37:44 -0600
> 
> In my company's code base, we have a lot of XML files that contain file
> names in tags.  For example:
> 
>   <file>foo/bar.txt</file>
> 
> Currently, ffap.el does not recognize these file names as such, because
> it recognizes the angle brackets as part of the file name.  This patch
> fixes that.
> ---
>  lisp/ffap.el | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/lisp/ffap.el b/lisp/ffap.el
> index 5383f743878..b2b681b7c44 100644
> --- a/lisp/ffap.el
> +++ b/lisp/ffap.el
> @@ -1065,6 +1065,9 @@ ffap-string-at-point-mode-alist
>      ;; (La)TeX: don't allow braces
>      (latex-mode "--:\\\\$+<>@-Z_[:alpha:]~*?" "<@" "@>;.,!:")
>      (tex-mode "--:\\\\$+<>@-Z_[:alpha:]~*?" "<@" "@>;.,!:")
> +    ;; XML: don't allow angle brackets
> +    (xml-mode "--:\\\\${}+@-Z_[:alpha:]~*?#" "{<@" "@>;.,!:}")
> +    (nxml-mode "--:\\\\${}+@-Z_[:alpha:]~*?#" "{<@" "@>;.,!:}")
>      )

Wouldn't it be better to teach ffap.el to use the <file>...</file>
markup directly in XML?  XML actually tells us this is a file name, so
why do we need to guess using regexps?

Thanks.

Re: [PATCH] ffap.el: Exclude angle brackets from file names in XML

[LemonBreezes]

[-- Attachment #1: Type: text/plain, Size: 1417 bytes --]

My bad, it doesn't actually say <file>, it says <recorddeds>, <bagdescription>, <lookupFolder>, etc. 

StrawberryTea

On Sat, Mar 9, 2024, at 11:48 PM, Eli Zaretskii wrote:
> > From: StrawberryTea <look@strawberrytea.xyz>
> > Date: Sat, 9 Mar 2024 15:37:44 -0600
> > 
> > In my company's code base, we have a lot of XML files that contain file
> > names in tags.  For example:
> > 
> >   <file>foo/bar.txt</file>
> > 
> > Currently, ffap.el does not recognize these file names as such, because
> > it recognizes the angle brackets as part of the file name.  This patch
> > fixes that.
> > ---
> >  lisp/ffap.el | 3 +++
> >  1 file changed, 3 insertions(+)
> > 
> > diff --git a/lisp/ffap.el b/lisp/ffap.el
> > index 5383f743878..b2b681b7c44 100644
> > --- a/lisp/ffap.el
> > +++ b/lisp/ffap.el
> > @@ -1065,6 +1065,9 @@ ffap-string-at-point-mode-alist
> >      ;; (La)TeX: don't allow braces
> >      (latex-mode "--:\\\\$+<>@-Z_[:alpha:]~*?" "<@" "@>;.,!:")
> >      (tex-mode "--:\\\\$+<>@-Z_[:alpha:]~*?" "<@" "@>;.,!:")
> > +    ;; XML: don't allow angle brackets
> > +    (xml-mode "--:\\\\${}+@-Z_[:alpha:]~*?#" "{<@" "@>;.,!:}")
> > +    (nxml-mode "--:\\\\${}+@-Z_[:alpha:]~*?#" "{<@" "@>;.,!:}")
> >      )
> 
> Wouldn't it be better to teach ffap.el to use the <file>...</file>
> markup directly in XML?  XML actually tells us this is a file name, so
> why do we need to guess using regexps?
> 
> Thanks.
> 

[-- Attachment #2: Type: text/html, Size: 2502 bytes --]

Re: [PATCH] ffap.el: Exclude angle brackets from file names in XML

[Eli Zaretskii]

> Date: Sun, 10 Mar 2024 00:04:46 -0600
> From: LemonBreezes <look@strawberrytea.xyz>
> Cc: emacs-devel@gnu.org
> 
> My bad, it doesn't actually say <file>, it says <recorddeds>, <bagdescription>, <lookupFolder>, etc. 

If those tags still mean it's a file name, my question still stands.

Re: [PATCH] ffap.el: Exclude angle brackets from file names in XML

[Yuri Khan]

On Sun, 10 Mar 2024 at 13:41, Eli Zaretskii <eliz@gnu.org> wrote:

> > My bad, it doesn't actually say <file>, it says <recorddeds>, <bagdescription>, <lookupFolder>, etc.
>
> If those tags still mean it's a file name, my question still stands.

Tags do not inherently mean anything in XML. Only a specific XML-based
format definition gives them meaning, and there is no way for either a
doctype definition or an XML Schema to say “elements named X
definitely contain file names”. The closest is XML Schema’s anyURI
datatype, but it might refer not necessarily to a local file but
possibly to any other URI-addressed resource.

Re: [PATCH] ffap.el: Exclude angle brackets from file names in XML

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Sun, 10 Mar 2024 14:20:59 +0700
> Cc: LemonBreezes <look@strawberrytea.xyz>, emacs-devel@gnu.org
> 
> On Sun, 10 Mar 2024 at 13:41, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > > My bad, it doesn't actually say <file>, it says <recorddeds>, <bagdescription>, <lookupFolder>, etc.
> >
> > If those tags still mean it's a file name, my question still stands.
> 
> Tags do not inherently mean anything in XML. Only a specific XML-based
> format definition gives them meaning, and there is no way for either a
> doctype definition or an XML Schema to say “elements named X
> definitely contain file names”. The closest is XML Schema’s anyURI
> datatype, but it might refer not necessarily to a local file but
> possibly to any other URI-addressed resource.

Are you saying that applications using XML hard-code the meaning of
each tag that they need to process?  IOW, how does a program using
such an XML know that the value is a file name?

Re: [PATCH] ffap.el: Exclude angle brackets from file names in XML

[Yuri Khan]

On Sun, 10 Mar 2024 at 14:51, Eli Zaretskii <eliz@gnu.org> wrote:

> > Tags do not inherently mean anything in XML. Only a specific XML-based
> > format definition gives them meaning, and there is no way for either a
> > doctype definition or an XML Schema to say “elements named X
> > definitely contain file names”. The closest is XML Schema’s anyURI
> > datatype, but it might refer not necessarily to a local file but
> > possibly to any other URI-addressed resource.
>
> Are you saying that applications using XML hard-code the meaning of
> each tag that they need to process?  IOW, how does a program using
> such an XML know that the value is a file name?

XML is a generic framework for designing formats. Each application
designs a specific schema that defines which tags mean what.

For example, XHTML is an XML subtype in which the <img> element can
have a src="…" attribute which contains an URI. In particular, it
might contain a file name.

For another example, SVG is an XML subtype that does not define an
<img> element; <img> is invalid if used in an SVG document. On the
other hand, SVG has an <image> element whose href="…" attribute
contains an URI.


A program typically does not work with arbitrary XML. It works with a
specific XML subtype, like OpenOffice with ODF (which is a ZIP archive
containing, among others, some XML files) and Inkscape with SVG. In
case an application is capable of understanding multiple XML-based
formats, it must know beforehand which particular format a document is
an instance of.

For example, when Firefox receives an HTTP response with a
‘Content-Type: application/xhtml+xml’ header, it interprets the
response body as XHTML page; but with a ‘Content-Type: image/svg+xml’,
it processes the body as an SVG image.


(I did not expect you’d need a lecture on this.)

Re: [PATCH] ffap.el: Exclude angle brackets from file names in XML

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Sun, 10 Mar 2024 17:30:50 +0700
> Cc: look@strawberrytea.xyz, emacs-devel@gnu.org
> 
> On Sun, 10 Mar 2024 at 14:51, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > > Tags do not inherently mean anything in XML. Only a specific XML-based
> > > format definition gives them meaning, and there is no way for either a
> > > doctype definition or an XML Schema to say “elements named X
> > > definitely contain file names”. The closest is XML Schema’s anyURI
> > > datatype, but it might refer not necessarily to a local file but
> > > possibly to any other URI-addressed resource.
> >
> > Are you saying that applications using XML hard-code the meaning of
> > each tag that they need to process?  IOW, how does a program using
> > such an XML know that the value is a file name?
> 
> XML is a generic framework for designing formats. Each application
> designs a specific schema that defines which tags mean what.
> 
> For example, XHTML is an XML subtype in which the <img> element can
> have a src="…" attribute which contains an URI. In particular, it
> might contain a file name.
> 
> For another example, SVG is an XML subtype that does not define an
> <img> element; <img> is invalid if used in an SVG document. On the
> other hand, SVG has an <image> element whose href="…" attribute
> contains an URI.
> 
> 
> A program typically does not work with arbitrary XML. It works with a
> specific XML subtype, like OpenOffice with ODF (which is a ZIP archive
> containing, among others, some XML files) and Inkscape with SVG. In
> case an application is capable of understanding multiple XML-based
> formats, it must know beforehand which particular format a document is
> an instance of.
> 
> For example, when Firefox receives an HTTP response with a
> ‘Content-Type: application/xhtml+xml’ header, it interprets the
> response body as XHTML page; but with a ‘Content-Type: image/svg+xml’,
> it processes the body as an SVG image.
> 
> 
> (I did not expect you’d need a lecture on this.)

I don't, not the general lecture you gave above, anyway.  I know about
the XML schema.  I was surprised that you didn't (and AFAIU still
don't) suggest that we interpret the tags according to the schema.
AFAIR, we do have an XML mode which can parse the schema, so it seems
reasonable to expect that we could recognize file names using the
information in the schema without any guesswork.  If this is
impossible, please explain why not, but please be specific: refer to
what Emacs can know about an XML document using the existing XML
mode(s), and how that affects our ability to know which tags specify
file names.

Re: [PATCH] ffap.el: Exclude angle brackets from file names in XML

[Yuri Khan]

On Sun, 10 Mar 2024 at 17:46, Eli Zaretskii <eliz@gnu.org> wrote:

> > > > there is no way for either a
> > > > doctype definition or an XML Schema to say “elements named X
> > > > definitely contain file names”. The closest is XML Schema’s anyURI
> > > > datatype, but it might refer not necessarily to a local file but
> > > > possibly to any other URI-addressed resource.

> I was surprised that you didn't (and AFAIU still
> don't) suggest that we interpret the tags according to the schema.

I think I explained that: because there is nothing in the XML Schema
meta-schema that talks specifically of files and file names.

It might be possible to consider element content or attributes typed
as ‘anyURI’ to contain file names. It will incur false positives (when
an anyURI attribute actually contains a non-file URI) and false
negatives (when the schema of a document is not known to Emacs, or
when it does not specifically designate an element or attribute as
‘anyURI’ even though it contains file names in practice).

> AFAIR, we do have an XML mode which can parse the schema, so it seems
> reasonable to expect that we could recognize file names using the
> information in the schema without any guesswork.  If this is
> impossible, please explain why not, but please be specific: refer to
> what Emacs can know about an XML document using the existing XML
> mode(s), and how that affects our ability to know which tags specify
> file names.

I assume you’re referring to ‘rnc-validate-mode’ minor mode and the
few schemas it knows about out of the box?

The OP is working with an unspecified, possibly proprietary, XML-based
format. Likely, its schema is not included in Emacs, and possibly not
even readily available to the OP. If available, it might not be in the
RELAX NG Compact format, requiring conversion. If not available, the
OP would have to derive a schema from observed document instances.

Meanwhile, the generic approach “the longest sequence of characters
allowed in file names, subject to restrictions of your buffer’s
syntax, starting from the point where you type M-RET and extending
both ways, is potentially a file name”, is easy to implement and
usable without any preliminary setup, although probably biased towards
false positives (attempting to produce a file name when the element or
attribute is not intended as such).

Re: [PATCH] ffap.el: Exclude angle brackets from file names in XML

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Sun, 10 Mar 2024 19:17:27 +0700
> Cc: look@strawberrytea.xyz, emacs-devel@gnu.org
> 
> On Sun, 10 Mar 2024 at 17:46, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > I was surprised that you didn't (and AFAIU still
> > don't) suggest that we interpret the tags according to the schema.
> 
> I think I explained that: because there is nothing in the XML Schema
> meta-schema that talks specifically of files and file names.

Then how do programs that process such XML know where are file names?

> It might be possible to consider element content or attributes typed
> as ‘anyURI’ to contain file names. It will incur false positives (when
> an anyURI attribute actually contains a non-file URI) and false
> negatives (when the schema of a document is not known to Emacs, or
> when it does not specifically designate an element or attribute as
> ‘anyURI’ even though it contains file names in practice).

Is that what programs which process XML do?

> > AFAIR, we do have an XML mode which can parse the schema, so it seems
> > reasonable to expect that we could recognize file names using the
> > information in the schema without any guesswork.  If this is
> > impossible, please explain why not, but please be specific: refer to
> > what Emacs can know about an XML document using the existing XML
> > mode(s), and how that affects our ability to know which tags specify
> > file names.
> 
> I assume you’re referring to ‘rnc-validate-mode’ minor mode and the
> few schemas it knows about out of the box?
> 
> The OP is working with an unspecified, possibly proprietary, XML-based
> format. Likely, its schema is not included in Emacs, and possibly not
> even readily available to the OP. If available, it might not be in the
> RELAX NG Compact format, requiring conversion. If not available, the
> OP would have to derive a schema from observed document instances.

So how does rnc-validate-mode somehow always knows to indicate whether
a given XML file is valid or not?

> Meanwhile, the generic approach “the longest sequence of characters
> allowed in file names, subject to restrictions of your buffer’s
> syntax, starting from the point where you type M-RET and extending
> both ways, is potentially a file name”, is easy to implement and
> usable without any preliminary setup, although probably biased towards
> false positives (attempting to produce a file name when the element or
> attribute is not intended as such).

If that's the best (not necessarily the easiest) we can do, I'm okay
with doing that.  But in general, IMO ffap.el is full of problematic
heuristics that is known to fail from time to time, so whenever we can
avoid the guesswork, we should, even if it is not the easiest
solution.

Thanks.

Re: [PATCH] ffap.el: Exclude angle brackets from file names in XML

[Yuri Khan]

On Sun, 10 Mar 2024 at 19:43, Eli Zaretskii <eliz@gnu.org> wrote:

> > I think I explained that: because there is nothing in the XML Schema
> > meta-schema that talks specifically of files and file names.
>
> Then how do programs that process such XML know where are file names?

By inside knowledge that is more nuanced than can be expressed in a
schema. Or by the virtue of the fact that a relative file name is also
valid as a relative URI reference and resolving it against a base URI
that names a file yields an URI also naming a file.

> > It might be possible to consider element content or attributes typed
> > as ‘anyURI’ to contain file names. It will incur false positives (when
> > an anyURI attribute actually contains a non-file URI) and false
> > negatives (when the schema of a document is not known to Emacs, or
> > when it does not specifically designate an element or attribute as
> > ‘anyURI’ even though it contains file names in practice).
>
> Is that what programs which process XML do?

I cannot, off the top of my head, name any program which processes XML
in a generic way (i.e. not specialized to a particular subtype, guided
only by schemas) *and also* handles files referenced.


> So how does rnc-validate-mode somehow always knows to indicate whether
> a given XML file is valid or not?

It doesn’t always.

It attempts to heuristically detect the file’s schema by file name
extension, or by the namespace or local name of its root element. If
that fails, it applies what it calls a “vacuous schema” — one by which
any well-formed XML is valid, and by which none of the elements or
attributes are known to be anyURI.


> If that's the best (not necessarily the easiest) we can do, I'm okay
> with doing that.  But in general, IMO ffap.el is full of problematic
> heuristics that is known to fail from time to time, so whenever we can
> avoid the guesswork, we should, even if it is not the easiest
> solution.

No argument from me here, but as long as we have heuristics, might as
well apply improvements to them.

Re: [PATCH] ffap.el: Exclude angle brackets from file names in XML

[Lynn Winebarger]

[-- Attachment #1: Type: text/plain, Size: 768 bytes --]

On Sun, Mar 10, 2024, 9:17 AM Yuri Khan <yuri.v.khan@gmail.com> wrote:

> On Sun, 10 Mar 2024 at 19:43, Eli Zaretskii <eliz@gnu.org> wrote:
>
>
>
> > If that's the best (not necessarily the easiest) we can do, I'm okay
> > with doing that.  But in general, IMO ffap.el is full of problematic
> > heuristics that is known to fail from time to time, so whenever we can
> > avoid the guesswork, we should, even if it is not the easiest
> > solution.
>
> No argument from me here, but as long as we have heuristics, might as
> well apply improvements to them.
>

I'm ignorant here, but are angle brackets not in the syntax table as
parentheses for XML modes?   Using structural syntax to delineate file
names seems like more than a heuristic.

Lynn

[-- Attachment #2: Type: text/html, Size: 1442 bytes --]

Re: [PATCH] ffap.el: Exclude angle brackets from file names in XML

[Eli Zaretskii]

> From: StrawberryTea <look@strawberrytea.xyz>
> Date: Sat, 9 Mar 2024 15:37:44 -0600
> 
> In my company's code base, we have a lot of XML files that contain file
> names in tags.  For example:
> 
>   <file>foo/bar.txt</file>
> 
> Currently, ffap.el does not recognize these file names as such, because
> it recognizes the angle brackets as part of the file name.  This patch
> fixes that.

Thanks, installed.

This contribution gets you dangerously close to the limit of what we
can accept from you without copyright assignment.  Would you like to
start the paperwork of assigning the copyright at this time?

Also, in the future please try sending patches in the form of "git
format-patch", which makes it easier for us to install the changes.