bug#74907: 31.0.50; nsm-check-local-subnet-ipv4 test fails on macOS with VPN enabled

bug#74907: 31.0.50; nsm-check-local-subnet-ipv4 test fails on macOS with VPN enabled

[Stefan Kangas]

I see the below test failure when running with a VPN enabled on macOS.
It fails predictably every time, but when I disable the VPN, the test
passes.

Looking into it, it seems like it's this call that leads to the
backtrace:

    (nsm-should-check "localhost")

However, when edebugging `nsm-should-check` and step through the code, I
do not get a backtrace, and it correctly returns t.

Any ideas for how to continue debugging this?


Running 2 tests (2024-12-16 02:41:49+0100, selector ‘(not (or (tag
:expensive-test) (tag :unstable) (tag :nativecomp)))’)
Test nsm-check-local-subnet-ipv4 backtrace:
  signal(wrong-type-argument (arrayp (0 . [0 0 255 0 0 0 16 2 0 0 10 1
  signal(wrong-type-argument (arrayp (0 . [0 0 255 0 0 0 16 2 0 0 10 1
  apply(signal (wrong-type-argument (arrayp (0 . [0 0 255 0 0 0 16 2 0
  #f(compiled-function () #<bytecode 0xb0a4a42ef70c609>)()
  #f(compiled-function () #<bytecode 0x14ecd754278a5065>)()
  handler-bind-1(#f(compiled-function () #<bytecode 0x14ecd754278a5065
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name nsm-check-local-subnet-ipv4 :document
  ert-run-or-rerun-test(#s(ert--stats :selector (not (or (tag :expensi
  ert-run-tests((not (or (tag :expensive-test) (tag :unstable) (tag :n
  ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable) (
  ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
  eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
  command-line-1(("-L" ":." "-l" "ert" "--eval" "(setq treesit-extra-l
  command-line()
  normal-top-level()
Test nsm-check-local-subnet-ipv4 condition:
    (wrong-type-argument arrayp (0 . [0 0 255 0 0 0 16 2 0 0 ...]))
   FAILED  1/2  nsm-check-local-subnet-ipv4 (0.021312 sec) at
lisp/net/nsm-tests.el:30
  skipped  2/2  nsm-check-local-subnet-ipv6 (0.000127 sec)

Ran 2 tests, 0 results as expected, 1 unexpected, 1 skipped
(2024-12-16 02:41:49+0100, 0.108002 sec)

1 unexpected results:
   FAILED  nsm-check-local-subnet-ipv4

1 skipped results:
  SKIPPED  nsm-check-local-subnet-ipv6


In GNU Emacs 31.0.50 (build 17, aarch64-apple-darwin24.1.0, NS
 appkit-2575.20 Version 15.1.1 (Build 24B91)) of 2024-12-16 built on
 foo.local
Repository revision: 29058579e9f27872d47e9d5146dfd9ce79697a0d
Repository branch: master
Windowing system distributor 'Apple', version 10.3.2575
System Description:  macOS 15.1.1

Configured using:
 'configure --without-dbus'

bug#74907: 31.0.50; nsm-check-local-subnet-ipv4 test fails on macOS with VPN enabled

[Robert Pluim]

>>>>> On Mon, 16 Dec 2024 02:03:53 +0000, Stefan Kangas <stefankangas@gmail.com> said:

    Stefan> I see the below test failure when running with a VPN enabled on macOS.
    Stefan> It fails predictably every time, but when I disable the VPN, the test
    Stefan> passes.

    Stefan> Looking into it, it seems like it's this call that leads to the
    Stefan> backtrace:

    Stefan>     (nsm-should-check "localhost")

    Stefan> However, when edebugging `nsm-should-check` and step through the code, I
    Stefan> do not get a backtrace, and it correctly returns t.

    Stefan> Any ideas for how to continue debugging this?

`printf' (or in this case `message') is your friend :-)

I suspect `network-interface-list' is returning unexpected values
because of the VPN, but Iʼd check `network-lookup-address-info' as well

Robert
-- 

bug#74907: 31.0.50; nsm-check-local-subnet-ipv4 test fails on macOS with VPN enabled

[Ship Mints]

[-- Attachment #1: Type: text/plain, Size: 1456 bytes --]

With the VPN running and without, check the value
of (network-lookup-address-info "localhost") and see if the VPN-regime
value looks sensible. It is possible that your VPN set up is absconding
with host resolution. I'd bet that (nsm-should-check "127.0.0.1") works
fine under both scenarios. You could at the command line also ping
localhost and see what that reveals or try macOS network "reachability"
diagnostic utility scutil -W -r localhost.

On Mon, Dec 16, 2024 at 5:58 AM Robert Pluim <rpluim@gmail.com> wrote:

> >>>>> On Mon, 16 Dec 2024 02:03:53 +0000, Stefan Kangas <
> stefankangas@gmail.com> said:
>
>     Stefan> I see the below test failure when running with a VPN enabled
> on macOS.
>     Stefan> It fails predictably every time, but when I disable the VPN,
> the test
>     Stefan> passes.
>
>     Stefan> Looking into it, it seems like it's this call that leads to the
>     Stefan> backtrace:
>
>     Stefan>     (nsm-should-check "localhost")
>
>     Stefan> However, when edebugging `nsm-should-check` and step through
> the code, I
>     Stefan> do not get a backtrace, and it correctly returns t.
>
>     Stefan> Any ideas for how to continue debugging this?
>
> `printf' (or in this case `message') is your friend :-)
>
> I suspect `network-interface-list' is returning unexpected values
> because of the VPN, but Iʼd check `network-lookup-address-info' as well
>
> Robert
> --
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 2049 bytes --]

bug#74907: 31.0.50; nsm-check-local-subnet-ipv4 test fails on macOS with VPN enabled

[Robert Pluim]

>>>>> On Mon, 16 Dec 2024 06:23:12 -0500, Ship Mints <shipmints@gmail.com> said:

    Ship> With the VPN running and without, check the value
    Ship> of (network-lookup-address-info "localhost") and see if the VPN-regime
    Ship> value looks sensible. It is possible that your VPN set up is absconding
    Ship> with host resolution. I'd bet that (nsm-should-check "127.0.0.1") works
    Ship> fine under both scenarios. You could at the command line also ping
    Ship> localhost and see what that reveals or try macOS network "reachability"
    Ship> diagnostic utility scutil -W -r localhost.

Looking at the code, itʼs possible that `network-lookup-address-info'
needs a small adjustment. Letʼs see what Stefan says.

Robert
-- 

bug#74907: 31.0.50; nsm-check-local-subnet-ipv4 test fails on macOS with VPN enabled

[Stefan Kangas]

Ship Mints <shipmints@gmail.com> writes:

> With the VPN running and without, check the value
> of (network-lookup-address-info "localhost") and see if the VPN-regime
> value looks sensible. It is possible that your VPN set up is absconding
> with host resolution. I'd bet that (nsm-should-check "127.0.0.1") works
> fine under both scenarios.

I don't see a difference there.

;; With VPN
    (network-lookup-address-info "localhost")
    => ([0 0 0 0 0 0 0 1 0] [127 0 0 1 0])

    (network-lookup-address-info "127.0.0.1")
    => ([127 0 0 1 0])

;; Without VPN
    (network-lookup-address-info "localhost")
    => ([0 0 0 0 0 0 0 1 0] [127 0 0 1 0])

    (network-lookup-address-info "127.0.0.1")
    => ([127 0 0 1 0])

> You could at the command line also ping
> localhost and see what that reveals or try macOS network "reachability"
> diagnostic utility scutil -W -r localhost.

ping localhost works both with VPN and without, and scutil gives
basically the same output also.

bug#74907: 31.0.50; nsm-check-local-subnet-ipv4 test fails on macOS with VPN enabled

[Robert Pluim]

>>>>> On Mon, 16 Dec 2024 21:55:33 +0000, Stefan Kangas <stefankangas@gmail.com> said:

    Stefan> I don't see a difference there.

    Stefan> ;; With VPN
    Stefan>     (network-lookup-address-info "localhost")
    Stefan>     => ([0 0 0 0 0 0 0 1 0] [127 0 0 1 0])

    Stefan>     (network-lookup-address-info "127.0.0.1")
    Stefan>     => ([127 0 0 1 0])

    Stefan> ;; Without VPN
    Stefan>     (network-lookup-address-info "localhost")
    Stefan>     => ([0 0 0 0 0 0 0 1 0] [127 0 0 1 0])

    Stefan>     (network-lookup-address-info "127.0.0.1")
    Stefan>     => ([127 0 0 1 0])

In an interactive session or -batch?

Anyway, hereʼs a wild stab in the dark based on the only code path I
could see that would give your original output. If that works Iʼd like
to know which VPN client youʼre using so I can avoid it 😀

diff --git a/src/process.c b/src/process.c
index cd1378f07ad..7f14db31c43 100644
--- a/src/process.c
+++ b/src/process.c
@@ -4764,13 +4764,17 @@ DEFUN ("network-lookup-address-info", Fnetwork_lookup_address_info,
     {
       for (lres = res; lres; lres = lres->ai_next)
         {
-#ifndef AF_INET6
-          if (lres->ai_family != AF_INET)
-            continue;
+	  /* Avoid converting non-IP addresses (Bug#74907).  */
+	  if (lres->ai_family == AF_INET
+#ifdef AF_INET6
+	      || lres->ai_family == AF_INET6
 #endif
-          addresses = Fcons (conv_sockaddr_to_lisp (lres->ai_addr,
-                                                    lres->ai_addrlen),
-                             addresses);
+	      )
+	    addresses = Fcons (conv_sockaddr_to_lisp (lres->ai_addr,
+						      lres->ai_addrlen),
+			       addresses);
+	  else
+	    continue;
         }
       addresses = Fnreverse (addresses);
 
Robert
-- 

bug#74907: 31.0.50; nsm-check-local-subnet-ipv4 test fails on macOS with VPN enabled

[Stefan Kangas]

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Mon, 16 Dec 2024 21:55:33 +0000, Stefan Kangas <stefankangas@gmail.com> said:
>
>     Stefan> I don't see a difference there.
>
>     Stefan> ;; With VPN
>     Stefan>     (network-lookup-address-info "localhost")
>     Stefan>     => ([0 0 0 0 0 0 0 1 0] [127 0 0 1 0])
>
>     Stefan>     (network-lookup-address-info "127.0.0.1")
>     Stefan>     => ([127 0 0 1 0])
>
>     Stefan> ;; Without VPN
>     Stefan>     (network-lookup-address-info "localhost")
>     Stefan>     => ([0 0 0 0 0 0 0 1 0] [127 0 0 1 0])
>
>     Stefan>     (network-lookup-address-info "127.0.0.1")
>     Stefan>     => ([127 0 0 1 0])
>
> In an interactive session or -batch?

In an interactive session.  I tried now in batch too, using this:

./src/emacs -Q -batch -eval '(progn (princ (network-lookup-address-info
"localhost")) (terpri) (princ (network-lookup-address-info
"127.0.0.1")))'

;; With VPN
    ([0 0 0 0 0 0 0 1 0] [127 0 0 1 0])
    ([127 0 0 1 0])

;; Without VPN
    ([0 0 0 0 0 0 0 1 0] [127 0 0 1 0])
    ([127 0 0 1 0])

> Anyway, hereʼs a wild stab in the dark based on the only code path I
> could see that would give your original output. If that works Iʼd like
> to know which VPN client youʼre using so I can avoid it 😀

It didn't work, unfortunately.

However, I now see that I made a mistake in my original recipe to
reproduce this: I didn't bind 'nsm-trust-local-network'.

I can reproduce the backtrace consistently by evaluating this in an
interactive session:

    (let ((nsm-trust-local-network t))
      (should (eq t (nsm-should-check "example.org"))))

This allowed edebugging this, and I see that the backtrace comes from
nsm.el:238, where we do:

    (substring (nth 3 info) 0 -1)

With a VPN, `info` is bound to this when I get the backtrace:

    ("utun0"
     [10 0 0 1 0]
     (0 . [0 0 10 255 255 255 16 2 0 0 10 0 0 1])
     (0 . [0 0 255 0 0 0 16 2 0 0 10 0 0 1]))

Without a VPN, `info` is bound to this instead:

    ("utun6"
     [65153 0 0 0 6123 19123 32345 45123 0]
     [65153 0 0 0 65535 65535 65535 65535 0]
     [65535 65535 65535 65535 0 0 0 0 0])

Clearly, this will not work:

    (substring '(0 . [1 2 3 4]) 0 -1)

So the question is why `network-interface-list` would return such an
unusual value here.

bug#74907: 31.0.50; nsm-check-local-subnet-ipv4 test fails on macOS with VPN enabled

[Robert Pluim]

>>>>> On Tue, 17 Dec 2024 11:26:28 +0000, Stefan Kangas <stefankangas@gmail.com> said:

    Stefan>     (substring (nth 3 info) 0 -1)

    Stefan> With a VPN, `info` is bound to this when I get the backtrace:

    Stefan>     ("utun0"
    Stefan>      [10 0 0 1 0]
    Stefan>      (0 . [0 0 10 255 255 255 16 2 0 0 10 0 0 1])
    Stefan>      (0 . [0 0 255 0 0 0 16 2 0 0 10 0 0 1]))

    Stefan> Without a VPN, `info` is bound to this instead:

    Stefan>     ("utun6"
    Stefan>      [65153 0 0 0 6123 19123 32345 45123 0]
    Stefan>      [65153 0 0 0 65535 65535 65535 65535 0]
    Stefan>      [65535 65535 65535 65535 0 0 0 0 0])

    Stefan> Clearly, this will not work:

    Stefan>     (substring '(0 . [1 2 3 4]) 0 -1)

    Stefan> So the question is why `network-interface-list` would return such an
    Stefan> unusual value here.

Because getifaddrs is returning bogus info for the netmask. How about this:

diff --git a/src/process.c b/src/process.c
index cd1378f07ad..4fe16ad1a85 100644
--- a/src/process.c
+++ b/src/process.c
@@ -4351,6 +4351,7 @@ network_interface_list (bool full, unsigned short match)
 
       if (full)
         {
+	  it->ifa_netmask->sa_family = it->ifa_addr->sa_family;
           elt = Fcons (conv_sockaddr_to_lisp (it->ifa_netmask, len), elt);
           /* There is an it->ifa_broadaddr field, but its contents are
              unreliable, so always calculate the broadcast address from

Robert
-- 

bug#74907: 31.0.50; nsm-check-local-subnet-ipv4 test fails on macOS with VPN enabled

[Stefan Kangas]

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Tue, 17 Dec 2024 11:26:28 +0000, Stefan Kangas <stefankangas@gmail.com> said:
>
>     Stefan>     (substring (nth 3 info) 0 -1)
>
>     Stefan> With a VPN, `info` is bound to this when I get the backtrace:
>
>     Stefan>     ("utun0"
>     Stefan>      [10 0 0 1 0]
>     Stefan>      (0 . [0 0 10 255 255 255 16 2 0 0 10 0 0 1])
>     Stefan>      (0 . [0 0 255 0 0 0 16 2 0 0 10 0 0 1]))
>
>     Stefan> Without a VPN, `info` is bound to this instead:
>
>     Stefan>     ("utun6"
>     Stefan>      [65153 0 0 0 6123 19123 32345 45123 0]
>     Stefan>      [65153 0 0 0 65535 65535 65535 65535 0]
>     Stefan>      [65535 65535 65535 65535 0 0 0 0 0])
>
>     Stefan> Clearly, this will not work:
>
>     Stefan>     (substring '(0 . [1 2 3 4]) 0 -1)
>
>     Stefan> So the question is why `network-interface-list` would return such an
>     Stefan> unusual value here.
>
> Because getifaddrs is returning bogus info for the netmask. How about this:
>
> diff --git a/src/process.c b/src/process.c
> index cd1378f07ad..4fe16ad1a85 100644
> --- a/src/process.c
> +++ b/src/process.c
> @@ -4351,6 +4351,7 @@ network_interface_list (bool full, unsigned short match)
>
>        if (full)
>          {
> +	  it->ifa_netmask->sa_family = it->ifa_addr->sa_family;
>            elt = Fcons (conv_sockaddr_to_lisp (it->ifa_netmask, len), elt);
>            /* There is an it->ifa_broadaddr field, but its contents are
>               unreliable, so always calculate the broadcast address from

Thanks, that fixes the issue.  Tested with and without VPN on macOS.

bug#74907: 31.0.50; nsm-check-local-subnet-ipv4 test fails on macOS with VPN enabled

[Robert Pluim]

>>>>> On Tue, 17 Dec 2024 12:45:26 +0000, Stefan Kangas <stefankangas@gmail.com> said:

    Stefan> Thanks, that fixes the issue.  Tested with and without VPN on macOS.

Please let me know the name of the VPN client so that I can write a
suitably scathing commit message (although itʼs possible this is
macOSʼ fault).

Robert
-- 

bug#74907: 31.0.50; nsm-check-local-subnet-ipv4 test fails on macOS with VPN enabled

[Ship Mints]

[-- Attachment #1: Type: text/plain, Size: 923 bytes --]

Good catch. Bug could be either or both Apple or vendor +/- the VPN
implementation and if it also uses Network Extension Framework filters or
whatever. The BSD-derived source code in question in netinet hasn't really
changed appreciably in ages. Could also be an implied "6to4" bridge that
the VPN sets up that screws things up. With the VPN running, go go System
Preferences...Network...and you should be able to visually identify if
there's a virtual "port" or relay.

On Tue, Dec 17, 2024 at 7:57 AM Robert Pluim <rpluim@gmail.com> wrote:

> >>>>> On Tue, 17 Dec 2024 12:45:26 +0000, Stefan Kangas <
> stefankangas@gmail.com> said:
>
>     Stefan> Thanks, that fixes the issue.  Tested with and without VPN on
> macOS.
>
> Please let me know the name of the VPN client so that I can write a
> suitably scathing commit message (although itʼs possible this is
> macOSʼ fault).
>
> Robert
> --
>

[-- Attachment #2: Type: text/html, Size: 1403 bytes --]

bug#74907: 31.0.50; nsm-check-local-subnet-ipv4 test fails on macOS with VPN enabled

[Ship Mints]

[-- Attachment #1: Type: text/plain, Size: 1264 bytes --]

Oh, also it may matter if this is a macOS VPN profile or a vendor product
that has its own code. macOS supports IPsec and IKEv2 built in and some
vendor products merely enable and disable VPN profiles under these
protocols.

On Tue, Dec 17, 2024 at 8:11 AM Ship Mints <shipmints@gmail.com> wrote:

> Good catch. Bug could be either or both Apple or vendor +/- the VPN
> implementation and if it also uses Network Extension Framework filters or
> whatever. The BSD-derived source code in question in netinet hasn't really
> changed appreciably in ages. Could also be an implied "6to4" bridge that
> the VPN sets up that screws things up. With the VPN running, go go System
> Preferences...Network...and you should be able to visually identify if
> there's a virtual "port" or relay.
>
> On Tue, Dec 17, 2024 at 7:57 AM Robert Pluim <rpluim@gmail.com> wrote:
>
>> >>>>> On Tue, 17 Dec 2024 12:45:26 +0000, Stefan Kangas <
>> stefankangas@gmail.com> said:
>>
>>     Stefan> Thanks, that fixes the issue.  Tested with and without VPN on
>> macOS.
>>
>> Please let me know the name of the VPN client so that I can write a
>> suitably scathing commit message (although itʼs possible this is
>> macOSʼ fault).
>>
>> Robert
>> --
>>
>

[-- Attachment #2: Type: text/html, Size: 2062 bytes --]

bug#74907: 31.0.50; nsm-check-local-subnet-ipv4 test fails on macOS with VPN enabled

[Robert Pluim]

>>>>> On Tue, 17 Dec 2024 08:11:36 -0500, Ship Mints <shipmints@gmail.com> said:

    Ship> Good catch. Bug could be either or both Apple or vendor +/- the VPN
    Ship> implementation and if it also uses Network Extension Framework filters or
    Ship> whatever. The BSD-derived source code in question in netinet hasn't really
    Ship> changed appreciably in ages. Could also be an implied "6to4" bridge that
    Ship> the VPN sets up that screws things up. With the VPN running, go go System
    Ship> Preferences...Network...and you should be able to visually identify if
    Ship> there's a virtual "port" or relay.

I suspect vendor more than Apple, since I donʼt see such behaviour
with either of the two VPN clients I have on my macOS machine.

Robert
-- 

bug#74907: 31.0.50; nsm-check-local-subnet-ipv4 test fails on macOS with VPN enabled

[Stefan Kangas]

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Tue, 17 Dec 2024 12:45:26 +0000, Stefan Kangas <stefankangas@gmail.com> said:
>
>     Stefan> Thanks, that fixes the issue.  Tested with and without VPN on macOS.
>
> Please let me know the name of the VPN client so that I can write a
> suitably scathing commit message (although itʼs possible this is
> macOSʼ fault).

The client is Mullvad VPN, but without looking into it further, I'm not
confident if it is them or Apple that should take the blame here.

Their website is:
    https://mullvad.net/

Note that while the VPN itself is a paid service, the license of their
VPN client is GPLv3:
    https://github.com/mullvad/mullvadvpn-app/

If someone is able to trace this back to something that they did wrong,
I'd recommend sending them a bug report.  They are typically quite
responsive to issues.  I don't understand this issue well enough to do
that myself, and I unfortunately don't have the time to invest into it.

Thank you for the quick fix, Robert!

bug#74907: 31.0.50; nsm-check-local-subnet-ipv4 test fails on macOS with VPN enabled

[Robert Pluim]

>>>>> On Tue, 17 Dec 2024 16:16:55 -0800, Stefan Kangas <stefankangas@gmail.com> said:

    Stefan> Robert Pluim <rpluim@gmail.com> writes:
    >>>>>>> On Tue, 17 Dec 2024 12:45:26 +0000, Stefan Kangas <stefankangas@gmail.com> said:
    >> 
    Stefan> Thanks, that fixes the issue.  Tested with and without VPN on macOS.
    >> 
    >> Please let me know the name of the VPN client so that I can write a
    >> suitably scathing commit message (although itʼs possible this is
    >> macOSʼ fault).

    Stefan> The client is Mullvad VPN, but without looking into it further, I'm not
    Stefan> confident if it is them or Apple that should take the blame here.

The value of sa_family in the ifa_netmask field returned via
getifaddrs is not specified in the standard, so Iʼve had to put down
the hammer of correction, and humbly accept that our code has to
change. 😀

Robert
-- 

bug#74907: 31.0.50; nsm-check-local-subnet-ipv4 test fails on macOS with VPN enabled

[Robert Pluim]

tags 74907 fixed
close 74907 31.1
quit

Closing.
Committed as d8e8e1d5ed2

Robert
--