Returning argument string (regexp-quote)

Returning argument string (regexp-quote)

[Mattias Engdegård]

[-- Attachment #1: Type: text/plain, Size: 354 bytes --]

In most calls to 'regexp-quote', the argument doesn't actually need converting. It would be nice to avoid the allocation in that case. Patch.

More generally, do we need to preserve allocation for functions not specifically documented to return new strings? I can imagine there being code that depend on it by accident, but string mutation is rare.


[-- Attachment #2: 0001-Allow-regexp-quote-to-return-its-argument.patch --]
[-- Type: application/octet-stream, Size: 1127 bytes --]

From 30b34719db89f6094346e9dd4004509674e7021c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mattias=20Engdeg=C3=A5rd?= <mattiase@acm.org>
Date: Mon, 23 Sep 2019 18:56:30 +0200
Subject: [PATCH] Allow regexp-quote to return its argument

* src/search.c (Fregexp_quote): Only allocate a new string if needed.
---
 src/search.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/src/search.c b/src/search.c
index 1e57d2ecbe..fc9b85f413 100644
--- a/src/search.c
+++ b/src/search.c
@@ -3138,10 +3138,12 @@ DEFUN ("regexp-quote", Fregexp_quote, Sregexp_quote, 1, 1, 0,
     }
 
   Lisp_Object result
-    = make_specified_string (temp,
-			     SCHARS (string) + backslashes_added,
-			     out - temp,
-			     STRING_MULTIBYTE (string));
+    = (out - temp > SBYTES (string)
+       ? make_specified_string (temp,
+                                SCHARS (string) + backslashes_added,
+                                out - temp,
+                                STRING_MULTIBYTE (string))
+       : string);
   SAFE_FREE ();
   return result;
 }
-- 
2.21.0 (Apple Git-122)

Re: Returning argument string (regexp-quote)

[Andreas Schwab]

On Sep 24 2019, Mattias Engdegård <mattiase@acm.org> wrote:

> diff --git a/src/search.c b/src/search.c
> index 1e57d2ecbe..fc9b85f413 100644
> --- a/src/search.c
> +++ b/src/search.c
> @@ -3138,10 +3138,12 @@ DEFUN ("regexp-quote", Fregexp_quote, Sregexp_quote, 1, 1, 0,
>      }
>  
>    Lisp_Object result
> -    = make_specified_string (temp,
> -			     SCHARS (string) + backslashes_added,
> -			     out - temp,
> -			     STRING_MULTIBYTE (string));
> +    = (out - temp > SBYTES (string)

          backslashes_added > 0

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: Returning argument string (regexp-quote)

[Eli Zaretskii]

> Feedback-ID: mattiase@acm.or
> From: Mattias Engdegård <mattiase@acm.org>
> Date: Tue, 24 Sep 2019 13:28:15 +0200
> 
> More generally, do we need to preserve allocation for functions not specifically documented to return new strings? I can imagine there being code that depend on it by accident, but string mutation is rare.

I think the benefits from such general changes are barely tangible,
whereas the risk of breaking someone's (perhaps even ours) code are
real.  So I don't think we should do that, not in general.

Re: Returning argument string (regexp-quote)

[Mattias Engdegård]

24 sep. 2019 kl. 18.20 skrev Eli Zaretskii <eliz@gnu.org>:
> 
>> More generally, do we need to preserve allocation for functions not specifically documented to return new strings? I can imagine there being code that depend on it by accident, but string mutation is rare.
> 
> I think the benefits from such general changes are barely tangible,
> whereas the risk of breaking someone's (perhaps even ours) code are
> real.  So I don't think we should do that, not in general.

Very well, I shan't go on a spree. What about the specific case of regexp-quote then?
It was responsible for a sizeable portion of local consing in some (non-public) code I was profiling.
Would the change need a mention in NEWS, and/or its doc string?

Re: Returning argument string (regexp-quote)

[Paul Eggert]

On 9/24/19 10:44 AM, Mattias Engdegård wrote:
> What about the specific case of regexp-quote then?
> It was responsible for a sizeable portion of local consing in some (non-public) code I was profiling.
> Would the change need a mention in NEWS, and/or its doc string?

The performance case for regexp-quote is reasonably strong. This is 
similar to the 'format' function, which is called a lot, which does not 
allocate a string if its argument is simple, and where its callers are 
extremely unlikely to rely on its allocating a string in that case.

Changes should be mentioned in NEWS and the adjusted behavior should be 
mentioned in the manual; I don't think it needs to be in the doc string. 
Please see commit 2017-10-04T21:29:58Z!eggert@cs.ucla.edu for how the 
doc was done in the 'format' case.

Re: Returning argument string (regexp-quote)

[Eli Zaretskii]

> From: Mattias Engdegård <mattiase@acm.org>
> Date: Tue, 24 Sep 2019 19:44:21 +0200
> Cc: emacs-devel@gnu.org
> 
> What about the specific case of regexp-quote then?

I think we could do that in this case.

> It was responsible for a sizeable portion of local consing in some (non-public) code I was profiling.

"Sizable portion" meaning what, in quantitative terms?

> Would the change need a mention in NEWS, and/or its doc string?

I think it should be in NEWS and in the ELisp manual.  Doc string
doesn't necessarily have to say that (although it isn't a catastrophe
if it does), given its current language.

Thanks.

Re: Returning argument string (regexp-quote)

[Mattias Engdegård]

[-- Attachment #1: Type: text/plain, Size: 829 bytes --]

24 sep. 2019 kl. 21.47 skrev Eli Zaretskii <eliz@gnu.org>:
> 
>> It was responsible for a sizeable portion of local consing in some (non-public) code I was profiling.
> 
> "Sizable portion" meaning what, in quantitative terms?

I no longer have the data, maybe 25-40% in the part of the code that I was looking at. It's not uncommon to use 'regexp-quote' just in case.

> I think it should be in NEWS and in the ELisp manual.  Doc string
> doesn't necessarily have to say that (although it isn't a catastrophe
> if it does), given its current language.

Thanks -- the updated patch includes NEWS and manual amendments, as well as a slightly simplified condition. The manual didn't actually say anything about a new string being returned (in contrast to 'format' before 3db388b0bf), but a note about it won't hurt.


[-- Attachment #2: 0001-Allow-regexp-quote-to-return-its-argument.patch --]
[-- Type: application/octet-stream, Size: 2045 bytes --]

From 95dd4e043ea4635c44b656d0145c6bb67408e47f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mattias=20Engdeg=C3=A5rd?= <mattiase@acm.org>
Date: Mon, 23 Sep 2019 18:56:30 +0200
Subject: [PATCH] Allow regexp-quote to return its argument

* src/search.c (Fregexp_quote): Only allocate a new string if needed.
---
 doc/lispref/searching.texi |  3 +++
 etc/NEWS                   |  4 ++++
 src/search.c               | 10 ++++++----
 3 files changed, 13 insertions(+), 4 deletions(-)

diff --git a/doc/lispref/searching.texi b/doc/lispref/searching.texi
index 015871fab0..a4b6533412 100644
--- a/doc/lispref/searching.texi
+++ b/doc/lispref/searching.texi
@@ -1717,6 +1717,9 @@ Regexp Functions
  (concat "\\s-" (regexp-quote string) "\\s-"))
 @end group
 @end example
+
+The returned string may be @var{string} itself if it does not contain
+any special characters.
 @end defun
 
 @cindex optimize regexp
diff --git a/etc/NEWS b/etc/NEWS
index 3f7d4894df..49d733b8ac 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -2156,6 +2156,10 @@ valid event type.
 ---
 ** The obsolete package xesam.el (since Emacs 24) has been removed.
 
++++
+** 'regexp-quote' may return its argument string.
+If the argument needs no quoting, it can be returned instead of a copy.
+
 \f
 * Lisp Changes in Emacs 27.1
 
diff --git a/src/search.c b/src/search.c
index 1e57d2ecbe..9d95dcbca5 100644
--- a/src/search.c
+++ b/src/search.c
@@ -3138,10 +3138,12 @@ DEFUN ("regexp-quote", Fregexp_quote, Sregexp_quote, 1, 1, 0,
     }
 
   Lisp_Object result
-    = make_specified_string (temp,
-			     SCHARS (string) + backslashes_added,
-			     out - temp,
-			     STRING_MULTIBYTE (string));
+    = (backslashes_added > 0
+       ? make_specified_string (temp,
+                                SCHARS (string) + backslashes_added,
+                                out - temp,
+                                STRING_MULTIBYTE (string))
+       : string);
   SAFE_FREE ();
   return result;
 }
-- 
2.21.0 (Apple Git-122)

Re: Returning argument string (regexp-quote)

[Eli Zaretskii]

> From: Mattias Engdegård <mattiase@acm.org>
> Date: Wed, 25 Sep 2019 15:41:28 +0200
> Cc: emacs-devel <emacs-devel@gnu.org>, Paul Eggert <eggert@cs.ucla.edu>
> 
> Thanks -- the updated patch includes NEWS and manual amendments, as well as a slightly simplified condition. The manual didn't actually say anything about a new string being returned (in contrast to 'format' before 3db388b0bf), but a note about it won't hurt.

LGTM, thanks.

Compilation failure on macOS 10.15 Catalina

[Zach Pearson]

I think that Apple has changed the way the Xcode command line tools arrange headers on the system e.g. they are no longer put into /usr/include but exist in 

/Library/Developer/CommandLineTools/usr/include/

Because of this Emacs cannot find libxml/tree.h during the compilation process. 

I’ve found that adding this directory to ac_x_header_dirs in configure, then re-running ./configure, the trying to compile allows the program to build (the compiler output notes some deprecations but they’re just warnings). Is this a patch that should be upstream? 

Thanks for any guidance/feedback. I’m not sure if this is a permanent change and Apple intended for it to be that way or not. 

Zach 

Re: Compilation failure on macOS 10.15 Catalina

[mituharu]

> I think that Apple has changed the way the Xcode command line tools
> arrange headers on the system e.g. they are no longer put into
> /usr/include but exist in
>
> /Library/Developer/CommandLineTools/usr/include/
>
> Because of this Emacs cannot find libxml/tree.h during the compilation
> process.

The current configure script is supposed to handle this situation.
I suspect you have bogus libxml-2.0.pc referring to /usr/include.
What is the output of the following commands?

  $ pkg-config --cflags --libs "libxml-2.0 > 2.6.17"
  $ pkg-config --variable pcfiledir "libxml-2.0 > 2.6.17"

See also https://debbugs.gnu.org/cgi/bugreport.cgi?bug=32927

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Compilation failure on macOS 10.15 Catalina

[Zach Pearson]

Sorry for the double reply, I forgot to cc the mailing list. Whoops. 

To the first: 

-I/usr/local/Cellar/libxml2/2.9.9_2/include/libxml2 -L/usr/local/Cellar/libxml2/2.9.9_2/lib -lxml2

And the second:

/usr/local/opt/libxml2/lib/pkgconfig

> On 25 Sep 2019, at 15:29, mituharu@math.s.chiba-u.ac.jp wrote:
> 
>> I think that Apple has changed the way the Xcode command line tools
>> arrange headers on the system e.g. they are no longer put into
>> /usr/include but exist in
>> 
>> /Library/Developer/CommandLineTools/usr/include/
>> 
>> Because of this Emacs cannot find libxml/tree.h during the compilation
>> process.
> 
> The current configure script is supposed to handle this situation.
> I suspect you have bogus libxml-2.0.pc referring to /usr/include.
> What is the output of the following commands?
> 
>  $ pkg-config --cflags --libs "libxml-2.0 > 2.6.17"
>  $ pkg-config --variable pcfiledir "libxml-2.0 > 2.6.17"
> 
> See also https://debbugs.gnu.org/cgi/bugreport.cgi?bug=32927
> 
> 				     YAMAMOTO Mitsuharu
> 				mituharu@math.s.chiba-u.ac.jp
> 
> 
> 

Re: Compilation failure on macOS 10.15 Catalina

[mituharu]

> To the first:
>
> -I/usr/local/Cellar/libxml2/2.9.9_2/include/libxml2
> -L/usr/local/Cellar/libxml2/2.9.9_2/lib -lxml2
>
> And the second:
>
> /usr/local/opt/libxml2/lib/pkgconfig

Do you have header files and libraries in the directories
shown in the first output?

					YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Compilation failure on macOS 10.15 Catalina

[Zach Pearson]

No. I the first directory, the headers are one folder deeper in a directory fully called

/usr/local/Cellar/libxml2/2.9.9_2/include/libxml2/libxml

In the second, the libraries are present. 

> On 25 Sep 2019, at 17:47, mituharu@math.s.chiba-u.ac.jp wrote:
> 
>> To the first:
>> 
>> -I/usr/local/Cellar/libxml2/2.9.9_2/include/libxml2
>> -L/usr/local/Cellar/libxml2/2.9.9_2/lib -lxml2
>> 
>> And the second:
>> 
>> /usr/local/opt/libxml2/lib/pkgconfig
> 
> Do you have header files and libraries in the directories
> shown in the first output?
> 
> 					YAMAMOTO Mitsuharu
> 				mituharu@math.s.chiba-u.ac.jp
> 
> 
> 

Re: Compilation failure on macOS 10.15 Catalina

[mituharu]

> No. I the first directory, the headers are one folder deeper in a
> directory fully called
>
> /usr/local/Cellar/libxml2/2.9.9_2/include/libxml2/libxml
>
> In the second, the libraries are present.

Looks normal, and configure should find and use them.
Could you show the libxml2-related part of the contents of
config.log (generated by unmodified configure) ?

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Compilation failure on macOS 10.15 Catalina

[chad]

[-- Attachment #1: Type: text/plain, Size: 882 bytes --]

Files in /usr/local/Cellar are generally placed there/managed by homebrew,
aren't they?

On Wed, Sep 25, 2019 at 4:08 PM Zach Pearson <zach@zjp.codes> wrote:

> No. I the first directory, the headers are one folder deeper in a
> directory fully called
>
> /usr/local/Cellar/libxml2/2.9.9_2/include/libxml2/libxml
>
> In the second, the libraries are present.
>
> > On 25 Sep 2019, at 17:47, mituharu@math.s.chiba-u.ac.jp wrote:
> >
> >> To the first:
> >>
> >> -I/usr/local/Cellar/libxml2/2.9.9_2/include/libxml2
> >> -L/usr/local/Cellar/libxml2/2.9.9_2/lib -lxml2
> >>
> >> And the second:
> >>
> >> /usr/local/opt/libxml2/lib/pkgconfig
> >
> > Do you have header files and libraries in the directories
> > shown in the first output?
> >
> >                                       YAMAMOTO Mitsuharu
> >                               mituharu@math.s.chiba-u.ac.jp
> >
> >
> >
>
>
>

[-- Attachment #2: Type: text/html, Size: 1469 bytes --]

Re: Compilation failure on macOS 10.15 Catalina

[Zach Pearson]

[-- Attachment #1: Type: text/plain, Size: 1738 bytes --]

Chad: They are as far as I know. 

Mithuaru:

 5808 configure:17366: checking for libxml-2.0 > 2.6.17
 5809 configure:17373: $PKG_CONFIG --exists --print-errors "libxml-2.0 > 2.6.17"
 5810 configure:17376: $? = 0
 5811 configure:17390: $PKG_CONFIG --exists --print-errors "libxml-2.0 > 2.6.17"
 5812 configure:17393: $? = 0
 5813 configure:17431: result: yes
 5814 configure:17466: checking for htmlReadMemory in -lxml2
 5815 configure:17491: gcc -o conftest -g3 -O2     conftest.c -lxml2 -lxml2   >&5
 5816 configure:17491: $? = 0
 5817 configure:17500: result: yes
 5818 configure:17529: checking for maillock in -lmail

Thank you 

- Zach 

> On 27 Sep 2019, at 18:22, chad <yandros@gmail.com> wrote:
> 
> Files in /usr/local/Cellar are generally placed there/managed by homebrew, aren't they? 
> 
> On Wed, Sep 25, 2019 at 4:08 PM Zach Pearson <zach@zjp.codes> wrote:
> No. I the first directory, the headers are one folder deeper in a directory fully called
> 
> /usr/local/Cellar/libxml2/2.9.9_2/include/libxml2/libxml
> 
> In the second, the libraries are present. 
> 
> > On 25 Sep 2019, at 17:47, mituharu@math.s.chiba-u.ac.jp <mailto:mituharu@math.s.chiba-u.ac.jp> wrote:
> > 
> >> To the first:
> >> 
> >> -I/usr/local/Cellar/libxml2/2.9.9_2/include/libxml2
> >> -L/usr/local/Cellar/libxml2/2.9.9_2/lib -lxml2
> >> 
> >> And the second:
> >> 
> >> /usr/local/opt/libxml2/lib/pkgconfig
> > 
> > Do you have header files and libraries in the directories
> > shown in the first output?
> > 
> >                                       YAMAMOTO Mitsuharu
> >                               mituharu@math.s.chiba-u.ac.jp <mailto:mituharu@math.s.chiba-u.ac.jp>
> > 
> > 
> > 
> 
> 


[-- Attachment #2: Type: text/html, Size: 3581 bytes --]

Re: Compilation failure on macOS 10.15 Catalina

[YAMAMOTO Mitsuharu]

On Wed, 02 Oct 2019 12:59:51 +0900,
Zach Pearson wrote:
>
>  5808 configure:17366: checking for libxml-2.0 > 2.6.17
>  5809 configure:17373: $PKG_CONFIG --exists --print-errors "libxml-2.0 > 2.6.17"
>  5810 configure:17376: $? = 0
>  5811 configure:17390: $PKG_CONFIG --exists --print-errors "libxml-2.0 > 2.6.17"
>  5812 configure:17393: $? = 0
>  5813 configure:17431: result: yes
>  5814 configure:17466: checking for htmlReadMemory in -lxml2
>  5815 configure:17491: gcc -o conftest -g3 -O2     conftest.c -lxml2 -lxml2   >&5
>  5816 configure:17491: $? = 0
>  5817 configure:17500: result: yes
>  5818 configure:17529: checking for maillock in -lmail

There seems to be no problem here.  What is the error message on
compilation in the first place?

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Compilation failure on macOS 10.15 Catalina

[Zach Pearson]

Mitsuharu:

xml.c:26:10: fatal error: 'libxml/tree.h' file not found
#include <libxml/tree.h>

Re: Compilation failure on macOS 10.15 Catalina

[YAMAMOTO Mitsuharu]

On Wed, 02 Oct 2019 13:34:46 +0900,
Zach Pearson wrote:
> 
> Mitsuharu:
> 
> xml.c:26:10: fatal error: 'libxml/tree.h' file not found
> #include <libxml/tree.h>

Please try "make V=1" and show the output around the error.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Compilation failure on macOS 10.15 Catalina

[Zach Pearson]

/Applications/Xcode.app/Contents/Developer/usr/bin/make -C lib all
make[1]: Nothing to be done for `all'.
/Applications/Xcode.app/Contents/Developer/usr/bin/make -C lib-src all
make[1]: Nothing to be done for `all'.
/Applications/Xcode.app/Contents/Developer/usr/bin/make -C src VCSWITNESS='$(srcdir)/../.git/logs/HEAD' all
gcc -c  -Demacs  -I. -I. -I../lib -I../lib         -I/usr/include/libxml2       -I/usr/local/Cellar/little-cms2/2.9/include       -MMD -MF deps/xml.d -MP   -I/usr/local/Cellar/gnutls/3.6.9/include -I/usr/local/Cellar/nettle/3.4.1/include -I/usr/local/Cellar/libtasn1/4.14/include -I/usr/local/Cellar/p11-kit/0.23.17/include/p11-kit-1    -Wno-switch -Wno-pointer-sign -Wno-string-plus-int -Wno-unknown-attributes -Wno-initializer-overrides -Wno-tautological-compare -Wno-tautological-constant-out-of-range-compare -g3 -O2  xml.c
xml.c:26:10: fatal error: 'libxml/tree.h' file not found
#include <libxml/tree.h>
         ^~~~~~~~~~~~~~~
1 error generated.
make[1]: *** [xml.o] Error 1
make: *** [src] Error 2

> On 1 Oct 2019, at 23:42, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> wrote:
> 
> On Wed, 02 Oct 2019 13:34:46 +0900,
> Zach Pearson wrote:
>> 
>> Mitsuharu:
>> 
>> xml.c:26:10: fatal error: 'libxml/tree.h' file not found
>> #include <libxml/tree.h>
> 
> Please try "make V=1" and show the output around the error.
> 
> 				     YAMAMOTO Mitsuharu
> 				mituharu@math.s.chiba-u.ac.jp

Re: Compilation failure on macOS 10.15 Catalina

[YAMAMOTO Mitsuharu]

On Wed, 02 Oct 2019 13:43:55 +0900,
Zach Pearson wrote:
> 
> /Applications/Xcode.app/Contents/Developer/usr/bin/make -C lib all
> make[1]: Nothing to be done for `all'.
> /Applications/Xcode.app/Contents/Developer/usr/bin/make -C lib-src all
> make[1]: Nothing to be done for `all'.
> /Applications/Xcode.app/Contents/Developer/usr/bin/make -C src VCSWITNESS='$(srcdir)/../.git/logs/HEAD' all
> gcc -c  -Demacs  -I. -I. -I../lib -I../lib         -I/usr/include/libxml2       -I/usr/local/Cellar/little-cms2/2.9/include       -MMD -MF deps/xml.d -MP   -I/usr/local/Cellar/gnutls/3.6.9/include -I/usr/local/Cellar/nettle/3.4.1/include -I/usr/local/Cellar/libtasn1/4.14/include -I/usr/local/Cellar/p11-kit/0.23.17/include/p11-kit-1    -Wno-switch -Wno-pointer-sign -Wno-string-plus-int -Wno-unknown-attributes -Wno-initializer-overrides -Wno-tautological-compare -Wno-tautological-constant-out-of-range-compare -g3 -O2  xml.c
> xml.c:26:10: fatal error: 'libxml/tree.h' file not found
> #include <libxml/tree.h>
>          ^~~~~~~~~~~~~~~
> 1 error generated.
> make[1]: *** [xml.o] Error 1
> make: *** [src] Error 2

-I/usr/include/libxml2 looks strange.  What is the output of
"grep LIBXML2_CFLAGS config.log" and
"grep /usr/include/libxml2 config.log"?

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Compilation failure on macOS 10.15 Catalina

[Zach Pearson]

[-- Attachment #1: Type: text/plain, Size: 1727 bytes --]

grep LIBXML2_CFLAGS config.log
ac_cv_env_LIBXML2_CFLAGS_set=
ac_cv_env_LIBXML2_CFLAGS_value=
pkg_cv_LIBXML2_CFLAGS=-I/usr/include/libxml2
LIBXML2_CFLAGS='-I/usr/include/libxml2’

There is no folder /usr/include/libxml2

> On 2 Oct 2019, at 00:00, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> wrote:
> 
> On Wed, 02 Oct 2019 13:43:55 +0900,
> Zach Pearson wrote:
>> 
>> /Applications/Xcode.app/Contents/Developer/usr/bin/make -C lib all
>> make[1]: Nothing to be done for `all'.
>> /Applications/Xcode.app/Contents/Developer/usr/bin/make -C lib-src all
>> make[1]: Nothing to be done for `all'.
>> /Applications/Xcode.app/Contents/Developer/usr/bin/make -C src VCSWITNESS='$(srcdir)/../.git/logs/HEAD' all
>> gcc -c  -Demacs  -I. -I. -I../lib -I../lib         -I/usr/include/libxml2       -I/usr/local/Cellar/little-cms2/2.9/include       -MMD -MF deps/xml.d -MP   -I/usr/local/Cellar/gnutls/3.6.9/include -I/usr/local/Cellar/nettle/3.4.1/include -I/usr/local/Cellar/libtasn1/4.14/include -I/usr/local/Cellar/p11-kit/0.23.17/include/p11-kit-1    -Wno-switch -Wno-pointer-sign -Wno-string-plus-int -Wno-unknown-attributes -Wno-initializer-overrides -Wno-tautological-compare -Wno-tautological-constant-out-of-range-compare -g3 -O2  xml.c
>> xml.c:26:10: fatal error: 'libxml/tree.h' file not found
>> #include <libxml/tree.h>
>>         ^~~~~~~~~~~~~~~
>> 1 error generated.
>> make[1]: *** [xml.o] Error 1
>> make: *** [src] Error 2
> 
> -I/usr/include/libxml2 looks strange.  What is the output of
> "grep LIBXML2_CFLAGS config.log" and
> "grep /usr/include/libxml2 config.log"?
> 
> 				     YAMAMOTO Mitsuharu
> 				mituharu@math.s.chiba-u.ac.jp <mailto:mituharu@math.s.chiba-u.ac.jp>

[-- Attachment #2: Type: text/html, Size: 11584 bytes --]

Re: Compilation failure on macOS 10.15 Catalina

[Zach Pearson]

grep LIBXML2_CFLAGS config.log
ac_cv_env_LIBXML2_CFLAGS_set=3D
ac_cv_env_LIBXML2_CFLAGS_value=3D
pkg_cv_LIBXML2_CFLAGS=3D-I/usr/include/libxml2
LIBXML2_CFLAGS=3D'-I/usr/include/libxml2=E2=80=99

There is no folder /usr/include/libxml2

> On 2 Oct 2019, at 00:00, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> wrote:
> 
> On Wed, 02 Oct 2019 13:43:55 +0900,
> Zach Pearson wrote:
>> 
>> /Applications/Xcode.app/Contents/Developer/usr/bin/make -C lib all
>> make[1]: Nothing to be done for `all'.
>> /Applications/Xcode.app/Contents/Developer/usr/bin/make -C lib-src all
>> make[1]: Nothing to be done for `all'.
>> /Applications/Xcode.app/Contents/Developer/usr/bin/make -C src VCSWITNESS='$(srcdir)/../.git/logs/HEAD' all
>> gcc -c  -Demacs  -I. -I. -I../lib -I../lib         -I/usr/include/libxml2       -I/usr/local/Cellar/little-cms2/2.9/include       -MMD -MF deps/xml.d -MP   -I/usr/local/Cellar/gnutls/3.6.9/include -I/usr/local/Cellar/nettle/3.4.1/include -I/usr/local/Cellar/libtasn1/4.14/include -I/usr/local/Cellar/p11-kit/0.23.17/include/p11-kit-1    -Wno-switch -Wno-pointer-sign -Wno-string-plus-int -Wno-unknown-attributes -Wno-initializer-overrides -Wno-tautological-compare -Wno-tautological-constant-out-of-range-compare -g3 -O2  xml.c
>> xml.c:26:10: fatal error: 'libxml/tree.h' file not found
>> #include <libxml/tree.h>
>>         ^~~~~~~~~~~~~~~
>> 1 error generated.
>> make[1]: *** [xml.o] Error 1
>> make: *** [src] Error 2
> 
> -I/usr/include/libxml2 looks strange.  What is the output of
> "grep LIBXML2_CFLAGS config.log" and
> "grep /usr/include/libxml2 config.log"?
> 
> 				     YAMAMOTO Mitsuharu
> 				mituharu@math.s.chiba-u.ac.jp

Re: Compilation failure on macOS 10.15 Catalina

[Xu Xin]

On Wed, Oct 2, 2019 at 3:44 AM Zach Pearson <zach@zjp.codes> wrote:
>
> grep LIBXML2_CFLAGS config.log
> ac_cv_env_LIBXML2_CFLAGS_set=3D
> ac_cv_env_LIBXML2_CFLAGS_value=3D
> pkg_cv_LIBXML2_CFLAGS=3D-I/usr/include/libxml2
> LIBXML2_CFLAGS=3D'-I/usr/include/libxml2=E2=80=99
>
> There is no folder /usr/include/libxml2
>
> > On 2 Oct 2019, at 00:00, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> wrote:
> >
> > On Wed, 02 Oct 2019 13:43:55 +0900,
> > Zach Pearson wrote:
> >>
> >> /Applications/Xcode.app/Contents/Developer/usr/bin/make -C lib all
> >> make[1]: Nothing to be done for `all'.
> >> /Applications/Xcode.app/Contents/Developer/usr/bin/make -C lib-src all
> >> make[1]: Nothing to be done for `all'.
> >> /Applications/Xcode.app/Contents/Developer/usr/bin/make -C src VCSWITNESS='$(srcdir)/../.git/logs/HEAD' all
> >> gcc -c  -Demacs  -I. -I. -I../lib -I../lib         -I/usr/include/libxml2       -I/usr/local/Cellar/little-cms2/2.9/include       -MMD -MF deps/xml.d -MP   -I/usr/local/Cellar/gnutls/3.6.9/include -I/usr/local/Cellar/nettle/3.4.1/include -I/usr/local/Cellar/libtasn1/4.14/include -I/usr/local/Cellar/p11-kit/0.23.17/include/p11-kit-1    -Wno-switch -Wno-pointer-sign -Wno-string-plus-int -Wno-unknown-attributes -Wno-initializer-overrides -Wno-tautological-compare -Wno-tautological-constant-out-of-range-compare -g3 -O2  xml.c
> >> xml.c:26:10: fatal error: 'libxml/tree.h' file not found
> >> #include <libxml/tree.h>
> >>         ^~~~~~~~~~~~~~~
> >> 1 error generated.
> >> make[1]: *** [xml.o] Error 1
> >> make: *** [src] Error 2
> >
> > -I/usr/include/libxml2 looks strange.  What is the output of
> > "grep LIBXML2_CFLAGS config.log" and
> > "grep /usr/include/libxml2 config.log"?
> >
> >                                    YAMAMOTO Mitsuharu
> >                               mituharu@math.s.chiba-u.ac.jp
>
>

This patch may works, it let pkgconfig able to recognizes libxml2
shipping with macOS:

diff --git a/Library/Homebrew/os/mac/pkgconfig/10.15/libxml-2.0.pc
b/Library/Homebrew/os/mac/pkgconfig/10.15/libxml-2.0.pc
index c297c6b45..22b2e69e8 100644
--- a/Library/Homebrew/os/mac/pkgconfig/10.15/libxml-2.0.pc
+++ b/Library/Homebrew/os/mac/pkgconfig/10.15/libxml-2.0.pc
@@ -1,7 +1,8 @@
 prefix=/usr
+xcode_prefix=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk${prefix}
 exec_prefix=${prefix}
 libdir=${exec_prefix}/lib
-includedir=${prefix}/include
+includedir=${xcode_prefix}/include
 modules=1

 Name: libXML

- XIn

Re: Compilation failure on macOS 10.15 Catalina

[Robert Pluim]

>>>>> On Wed, 25 Sep 2019 14:55:03 -0500, Zach Pearson <zach@zjp.codes> said:

    Zach> I think that Apple has changed the way the Xcode command line tools arrange headers on the system e.g. they are no longer put into /usr/include but exist in 
    Zach> /Library/Developer/CommandLineTools/usr/include/

    Zach> Because of this Emacs cannot find libxml/tree.h during the compilation process. 

    Zach> I’ve found that adding this directory to ac_x_header_dirs in
    Zach> configure, then re-running ./configure, the trying to compile allows
    Zach> the program to build (the compiler output notes some deprecations but
    Zach> they’re just warnings). Is this a patch that should be upstream?

    Zach> Thanks for any guidance/feedback. I’m not sure if this is a permanent change and Apple intended for it to be that way or not. 

When this happens, which seems to be with every new Xcode release, I do

     $ cd /Library/Developer/CommandLineTools/Packages/
     $ ls
     macOS_SDK_headers_for_macOS_10.14.pkg

And then I run that .pkg file, which ends up producing:

    /usr/include/libxml2/libxml/tree.h

Robert

Re: Compilation failure on macOS 10.15 Catalina

[Zach Pearson]

Robert: There does not appear to be a Packages directory this time around, but I’m open to the idea that my copy of Catalina is borked. 

Mitsuharu: Can you be more specific? Searching for libxml produces ~900 matches. 

> On 26 Sep 2019, at 03:34, Robert Pluim <rpluim@gmail.com> wrote:
> 
>>>>>> On Wed, 25 Sep 2019 14:55:03 -0500, Zach Pearson <zach@zjp.codes> said:
> 
>    Zach> I think that Apple has changed the way the Xcode command line tools arrange headers on the system e.g. they are no longer put into /usr/include but exist in 
>    Zach> /Library/Developer/CommandLineTools/usr/include/
> 
>    Zach> Because of this Emacs cannot find libxml/tree.h during the compilation process. 
> 
>    Zach> I’ve found that adding this directory to ac_x_header_dirs in
>    Zach> configure, then re-running ./configure, the trying to compile allows
>    Zach> the program to build (the compiler output notes some deprecations but
>    Zach> they’re just warnings). Is this a patch that should be upstream?
> 
>    Zach> Thanks for any guidance/feedback. I’m not sure if this is a permanent change and Apple intended for it to be that way or not. 
> 
> When this happens, which seems to be with every new Xcode release, I do
> 
>     $ cd /Library/Developer/CommandLineTools/Packages/
>     $ ls
>     macOS_SDK_headers_for_macOS_10.14.pkg
> 
> And then I run that .pkg file, which ends up producing:
> 
>    /usr/include/libxml2/libxml/tree.h
> 
> Robert
> 

Re: Compilation failure on macOS 10.15 Catalina

[mituharu]

> Mitsuharu: Can you be more specific? Searching for libxml produces ~900
> matches.

The lines between "checking for libxml-2.0 > 2.6.17" and
"checking for maillock in -lmail" in config.log.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp