bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Thomas Fitzsimmons]

[-- Attachment #1: Type: text/plain, Size: 1686 bytes --]

Hi,

An Excorporate user reported an issue when running on Emacs 26.1 that
didn't happen in Emacs 25.3.

The error message, when trying to retrieve calendar items on a given day
is:

soap-find-operation: Wrong type argument: soap-binding, ("http://schemas.microsoft.com/exchange/services/2006/messages" . "ExchangeServiceBinding")

The simplest workaround is:

   M-x package-reinstall RET excorporate RET

I narrowed the cause down to what seems to be an incompatibility between
25.x bytecode and 26.1.  This issue only happens when excorporate.elc
has been byte compiled in Emacs 25.x (the .x doesn't seem to matter; I
replicated it on Emacs 25.1) and is then run in Emacs 26.1.  It doesn't
happen with a fresh package install on 25.x or 26.1.

When in the failure state, if I re-eval this function:

(defun exco--bind-wsdl (wsdl service-url port-name target-namespace
                             binding-name)
  "Create a WSDL binding.
Create a binding port for WSDL from SERVICE-URL, PORT-NAME,
TARGET-NAMESPACE and BINDING-NAME."
  (let* ((namespace (soap-wsdl-find-namespace target-namespace wsdl))
         (port (make-soap-port
                :name port-name
                :binding (cons target-namespace binding-name)
                :service-url service-url)))
    (soap-namespace-put port namespace)
    (push port (soap-wsdl-ports wsdl))
    (soap-resolve-references port wsdl)
    wsdl))

everything works.

In terms higher level functionality, it seems like the call to
soap-resolve-references has no effect.

I'm attaching the 25.1 and 26.1 bytecode for this function,
disassembled.  Maybe someone can identify what's wrong or suggest next
steps?

Thank you,
Thomas


[-- Attachment #2: exco--bind-wsdl-dump-25.1.txt --]
[-- Type: text/plain, Size: 908 bytes --]

byte code for exco--bind-wsdl:
  doc:  Create a WSDL binding. ...
  args: (arg1 arg2 arg3 arg4 arg5)
0	constant  soap-wsdl-find-namespace
1	stack-ref 2
2	stack-ref 6
4	call	  2
5	constant  vector
6	constant  cl-struct-soap-port
7	stack-ref 5
8	constant  nil
9	stack-ref 8
11	stack-ref 7
13	stack-ref 7
15	cons	  
16	call	  5
17	constant  soap-namespace-put
18	stack-ref 1
19	stack-ref 3
20	call	  2
21	discard	  
22	stack-ref 6
24	constant  0
25	aref	  
26	varref	  cl-struct-soap-wsdl-tags
27	memq	  
28	goto-if-not-nil 1
31	constant  signal
32	constant  wrong-type-argument
33	constant  soap-wsdl
34	stack-ref 9
36	list2	  
37	call	  2
38	discard	  
39:1	stack-ref 6
41	dup	  
42	constant  4
43	stack-ref 3
44	stack-ref 3
45	constant  4
46	aref	  
47	cons	  
48	aset	  
49	discardN  2
51	constant  soap-resolve-references
52	stack-ref 1
53	stack-ref 8
55	call	  2
56	discard	  
57	stack-ref 6
59	return	  

[-- Attachment #3: exco--bind-wsdl-dump-26.1.txt --]
[-- Type: text/plain, Size: 905 bytes --]

byte code for exco--bind-wsdl:
  doc:  Create a WSDL binding. ...
  args: (arg1 arg2 arg3 arg4 arg5)
0	constant  soap-wsdl-find-namespace
1	stack-ref 2
2	stack-ref 6
4	call	  2
5	constant  record
6	constant  soap-port
7	stack-ref 5
8	constant  nil
9	stack-ref 8
11	stack-ref 7
13	stack-ref 7
15	cons	  
16	call	  5
17	constant  soap-namespace-put
18	stack-ref 1
19	stack-ref 3
20	call	  2
21	discard	  
22	constant  type-of
23	stack-ref 7
25	call	  1
26	varref	  cl-struct-soap-wsdl-tags
27	memq	  
28	goto-if-not-nil 1
31	constant  signal
32	constant  wrong-type-argument
33	constant  soap-wsdl
34	stack-ref 9
36	list2	  
37	call	  2
38	discard	  
39:1	stack-ref 6
41	dup	  
42	constant  4
43	stack-ref 3
44	stack-ref 3
45	constant  4
46	aref	  
47	cons	  
48	aset	  
49	discardN  2
51	constant  soap-resolve-references
52	stack-ref 1
53	stack-ref 8
55	call	  2
56	discard	  
57	stack-ref 6
59	return	  

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Noam Postavsky]

Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:

> soap-find-operation: Wrong type argument: soap-binding, ("http://schemas.microsoft.com/exchange/services/2006/messages" . "ExchangeServiceBinding")

> byte code for exco--bind-wsdl:
>   doc:  Create a WSDL binding. ...
>   args: (arg1 arg2 arg3 arg4 arg5)
> 0	constant  soap-wsdl-find-namespace
> 1	stack-ref 2
> 2	stack-ref 6
> 4	call	  2
> 5	constant  vector
> 6	constant  cl-struct-soap-port

> byte code for exco--bind-wsdl:
>   doc:  Create a WSDL binding. ...
>   args: (arg1 arg2 arg3 arg4 arg5)
> 0	constant  soap-wsdl-find-namespace
> 1	stack-ref 2
> 2	stack-ref 6
> 4	call	  2
> 5	constant  record
> 6	constant  soap-port

Looks like it's due to the change in how cl-struct works.  Does enabling
cl-old-struct-compat-mode help?

    cl-old-struct-compat-mode is an interactive autoloaded compiled Lisp
    function in ‘cl-lib.el’.

    (cl-old-struct-compat-mode &optional ARG)

    Enable backward compatibility with old-style structs.
    This can be needed when using code byte-compiled using the old
    macro-expansion of ‘cl-defstruct’ that used vectors objects instead
    of record objects.

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Thomas Fitzsimmons]

Noam Postavsky <npostavs@gmail.com> writes:

> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
>
>> soap-find-operation: Wrong type argument: soap-binding, ("http://schemas.microsoft.com/exchange/services/2006/messages" . "ExchangeServiceBinding")
>
>> byte code for exco--bind-wsdl:
>>   doc:  Create a WSDL binding. ...
>>   args: (arg1 arg2 arg3 arg4 arg5)
>> 0	constant  soap-wsdl-find-namespace
>> 1	stack-ref 2
>> 2	stack-ref 6
>> 4	call	  2
>> 5	constant  vector
>> 6	constant  cl-struct-soap-port
>
>> byte code for exco--bind-wsdl:
>>   doc:  Create a WSDL binding. ...
>>   args: (arg1 arg2 arg3 arg4 arg5)
>> 0	constant  soap-wsdl-find-namespace
>> 1	stack-ref 2
>> 2	stack-ref 6
>> 4	call	  2
>> 5	constant  record
>> 6	constant  soap-port
>
> Looks like it's due to the change in how cl-struct works.  Does enabling
> cl-old-struct-compat-mode help?
>
>     cl-old-struct-compat-mode is an interactive autoloaded compiled Lisp
>     function in ‘cl-lib.el’.
>
>     (cl-old-struct-compat-mode &optional ARG)
>
>     Enable backward compatibility with old-style structs.
>     This can be needed when using code byte-compiled using the old
>     macro-expansion of ‘cl-defstruct’ that used vectors objects instead
>     of record objects.

Hmm, that does look related, however I just retried with and without it.
That mode was already enabled when the failure happened (when I ran M-x
cl-old-struct-compat-mode, it output "Cl-Old-Struct-Compat mode
disabled").

If I disable the mode, then I get a different failure earlier:

(wrong-type-argument soap-element [cl-struct-soap-port "ExchangeServicePort" nil "https://mail.cisco.com/ews/exchange.asmx" ("http://schemas.microsoft.com/exchange/services/2006/messages" . "ExchangeServiceBinding")])

So it seems like cl-old-struct-compat-mode being enabled is having a
positive effect, but maybe it does not handle the specific
incompatibility I'm running into.

Thomas

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Noam Postavsky]

Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:

> So it seems like cl-old-struct-compat-mode being enabled is having a
> positive effect, but maybe it does not handle the specific
> incompatibility I'm running into.

Hmm, can't see anything else in the bytecode.  Can you post the full
backtrace from the failure?

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Thomas Fitzsimmons]

Noam Postavsky <npostavs@gmail.com> writes:

> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
>
>> So it seems like cl-old-struct-compat-mode being enabled is having a
>> positive effect, but maybe it does not handle the specific
>> incompatibility I'm running into.
>
> Hmm, can't see anything else in the bytecode.  Can you post the full
> backtrace from the failure?

Yes, but the error that produces the backtrace happens much later
via a different entry point, M-x exco-calendar-show-day:

Debugger entered--Lisp error: (wrong-type-argument soap-binding ("http://schemas.microsoft.com/exchange/services/2006/messages" . "ExchangeServiceBinding"))
  signal(wrong-type-argument (soap-binding ("http://schemas.microsoft.com/exchange/services/2006/messages" . "ExchangeServiceBinding")))
  soap-find-operation(#s(soap-port :name "ExchangeServicePort" :namespace-tag nil :service-url "https://redacted/ews/exchange.asmx" :binding ("http://schemas.microsoft.com/exchange/services/2006/messages" . "ExchangeServiceBinding")) "FindItem")
  soap-operation-arity(#s(soap-wsdl :origin "https://redacted/ews/exchange.asmx" :current-file nil :xmlschema-imports nil :ports (#s(soap-port :name "ExchangeServicePort" :namespace-tag nil :service-url "https://redacted/ews/exchange.asmx" :binding ("http://schemas.microsoft.com/exchange/services/2006/messages" . "ExchangeServiceBinding"))) :alias-table (("ns3" . "http://schemas.microsoft.com/exchange/services/2006/messages") ("ns2" . "http://schemas.microsoft.com/exchange/services/2006/types") ("ns1" . "urn:ExchangeServicePortType") ("soapenc" . "http://schemas.xmlsoap.org/soap/encoding/") ("xsd" . "http://www.w3.org/2001/XMLSchema")) :namespaces (#s(soap-namespace :name "urn:ExchangeServicePortType" :elements #<hash-table equal 294/325 0x9cf7109>) #s(soap-namespace :name "http://schemas.microsoft.com/exchange/services/2006/types" :elements #<hash-table equal 683/730 0x7b5ea85>) #s(soap-namespace :name "http://schemas.microsoft.com/exchange/services/2006/messages" :elements #<hash-table equal 745/1095 0x61383d9>) #s(soap-namespace :name "http://schemas.xmlsoap.org/soap/encoding/" :elements #<hash-table equal 33/65 0x6805be1>) #s(soap-namespace :name "http://www.w3.org/2001/XMLSchema" :elements #<hash-table equal 33/65 0x6eb5cfd>))) "ExchangeServicePort" "FindItem")
  exco-get-meetings-for-day(("fitzsim@cisco.com" . "https://redacted/ews/exchange.asmx") 6 7 2018 #f(compiled-function (&rest arguments) #<bytecode 0x6b449ed>))
  #f(compiled-function (identifier callback) #<bytecode 0x6b44bfd>)(("fitzsim@cisco.com" . "https://redacted/ews/exchange.asmx") #f(compiled-function (&rest arguments) #<bytecode 0x6b449ed>))
  exco-connection-iterate(exco-org-initialize-buffer #f(compiled-function (identifier callback) #<bytecode 0x6b44bfd>) exco-org-insert-meetings exco-org-finalize-buffer)
  exco-org-show-day(6 7 2018)
  apply(exco-org-show-day (6 7 2018))
  exco-calendar-show-day()
  funcall-interactively(exco-calendar-show-day)
  call-interactively(exco-calendar-show-day nil nil)
  command-execute(exco-calendar-show-day)

The initial problem in exco--bind-wsdl (via the M-x excorporate entry
point) does not throw an error.  Basically, in the failure case, parsing
of the WSDL XML is incomplete, and the later error is due to that.

I printed the value of port from 26.1; with 25.1 bytecode it is:

[cl-struct-soap-port ExchangeServicePort nil https://mail.cisco.com/ews/exchange.asmx (http://schemas.microsoft.com/exchange/services/2006/messages . ExchangeServiceBinding)]

and with 26.1 bytecode it is:

#s(soap-port ExchangeServicePort nil https://mail.cisco.com/ews/exchange.asmx (http://schemas.microsoft.com/exchange/services/2006/messages . ExchangeServiceBinding))

Thomas

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Noam Postavsky]

On 7 June 2018 at 10:06, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:

> Yes, but the error that produces the backtrace happens much later
> via a different entry point, M-x exco-calendar-show-day:
>
> Debugger entered--Lisp error: (wrong-type-argument soap-binding ("http://schemas.microsoft.com/exchange/services/2006/messages" . "ExchangeServiceBinding"))
>   signal(wrong-type-argument (soap-binding ("http://schemas.microsoft.com/exchange/services/2006/messages" . "ExchangeServiceBinding")))
>   soap-find-operation(#s(soap-port :name "ExchangeServicePort" :namespace-tag nil :service-url "https://redacted/ews/exchange.asmx" :binding ("http://schemas.microsoft.com/exchange/services/2006/messages" . "ExchangeServiceBinding")) "FindItem")

> The initial problem in exco--bind-wsdl (via the M-x excorporate entry
> point) does not throw an error.  Basically, in the failure case, parsing
> of the WSDL XML is incomplete, and the later error is due to that.

Right, if I just evaluate the following in 26.1 without any 25.1
bytecode I get a similar error:

(let ((port (make-soap-port
         :name "ExchangeServicePort"
         :namespace-tag nil
         :service-url "https://redacted/ews/exchange.asmx"
         :binding
'("http://schemas.microsoft.com/exchange/services/2006/messages"
            . "ExchangeServiceBinding"))))
  (soap-find-operation port "FindItem"))

I don't quite understand how it's supposed to work successfully
though. Is there a way to see it working without having to run a
server on the other end (i.e., can you give some elisp forms to
evaluate that make the above work)?

I'm guessing the problem is with some old and new structs not matching
according to `equal' and/or `sxhash', which cl-old-struct-compat-mode
doesn't do anything about.

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Thomas Fitzsimmons]

Noam Postavsky <npostavs@gmail.com> writes:

> On 7 June 2018 at 10:06, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
>
>> Yes, but the error that produces the backtrace happens much later
>> via a different entry point, M-x exco-calendar-show-day:
>>
>> Debugger entered--Lisp error: (wrong-type-argument soap-binding ("http://schemas.microsoft.com/exchange/services/2006/messages" . "ExchangeServiceBinding"))
>>   signal(wrong-type-argument (soap-binding ("http://schemas.microsoft.com/exchange/services/2006/messages" . "ExchangeServiceBinding")))
>>   soap-find-operation(#s(soap-port :name "ExchangeServicePort" :namespace-tag nil :service-url "https://redacted/ews/exchange.asmx" :binding ("http://schemas.microsoft.com/exchange/services/2006/messages" . "ExchangeServiceBinding")) "FindItem")
>
>> The initial problem in exco--bind-wsdl (via the M-x excorporate entry
>> point) does not throw an error.  Basically, in the failure case, parsing
>> of the WSDL XML is incomplete, and the later error is due to that.
>
> Right, if I just evaluate the following in 26.1 without any 25.1
> bytecode I get a similar error:
>
> (let ((port (make-soap-port
>          :name "ExchangeServicePort"
>          :namespace-tag nil
>          :service-url "https://redacted/ews/exchange.asmx"
>          :binding
> '("http://schemas.microsoft.com/exchange/services/2006/messages"
>             . "ExchangeServiceBinding"))))
>   (soap-find-operation port "FindItem"))
>
> I don't quite understand how it's supposed to work successfully
> though. Is there a way to see it working without having to run a
> server on the other end (i.e., can you give some elisp forms to
> evaluate that make the above work)?
>
> I'm guessing the problem is with some old and new structs not matching
> according to `equal' and/or `sxhash', which cl-old-struct-compat-mode
> doesn't do anything about.

I did some more narrowing and found that the issue is with the use of
symbol properties in soap-resolve-references.  I added debugging:

(defun soap-resolve-references (element wsdl)
  "Replace names in ELEMENT with the referenced objects in the WSDL.
This is a generic function which invokes a specific resolver
function depending on the type of the ELEMENT.

If ELEMENT has no resolver function, it is silently ignored."
  (when (equal (soap-element-name element) "ExchangeServicePort")
    (message "1: %s" element)
    (message "2: %s" (aref element 0))
    (message "3: %s" (get (aref element 0) 'soap-resolve-references)))
  (let ((resolver (get (aref element 0) 'soap-resolve-references)))
    (when resolver
      (funcall resolver element wsdl))))

In the working case, I get:

1: #s(soap-port ExchangeServicePort nil https://mail.cisco.com/ews/exchange.asmx (http://schemas.microsoft.com/exchange/services/2006/messages . ExchangeServiceBinding))
2: soap-port
3: soap-resolve-references-for-port

while in the failing case, I get:

1: [cl-struct-soap-port ExchangeServicePort nil https://mail.cisco.com/ews/exchange.asmx (http://schemas.microsoft.com/exchange/services/2006/messages . ExchangeServiceBinding)]
2: cl-struct-soap-port
3: nil

This means the resolver fails to run and the ExchangeServicePort element
is not expanded, which results in exco-calendar-show-day's subsequent
failure.

It seems like soap-client's use of symbol properties is not safe across
major Emacs versions.

Thomas

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Noam Postavsky]

Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:

> (defun soap-resolve-references (element wsdl)

>   (let ((resolver (get (aref element 0) 'soap-resolve-references)))

Ah, then it should work to replace the (aref element 0) with (type-of
element) as long as cl-old-struct-compat-mode is enabled.

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Thomas Fitzsimmons]

Noam Postavsky <npostavs@gmail.com> writes:

> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
>
>> (defun soap-resolve-references (element wsdl)
>
>>   (let ((resolver (get (aref element 0) 'soap-resolve-references)))
>
> Ah, then it should work to replace the (aref element 0) with (type-of
> element) as long as cl-old-struct-compat-mode is enabled.

Yes, I retested with that change and it does work.  I guess I should
submit a patch to change all those occurrences in soap-client.  I'm
hoping cl-old-struct-compat-mode will continue to be enabled by default
for a long time.  Thanks for looking into this.

Thomas

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Noam Postavsky]

Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:

> Noam Postavsky <npostavs@gmail.com> writes:
>
>> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
>>
>>> (defun soap-resolve-references (element wsdl)
>>
>>>   (let ((resolver (get (aref element 0) 'soap-resolve-references)))
>>
>> Ah, then it should work to replace the (aref element 0) with (type-of
>> element) as long as cl-old-struct-compat-mode is enabled.
>
> Yes, I retested with that change and it does work.  I guess I should
> submit a patch to change all those occurrences in soap-client.

I just noticed soap-client is also an ELPA package; then there is the
additional complication that type-of will give the wrong answer in Emacs
versions 25 and older.  So you would need something like

    (let ((type (type-of element)))
      (if (eq type 'vector)
          (aref element 0) ; For Emacs 25 and earlier.
        type))

> I'm hoping cl-old-struct-compat-mode will continue to be enabled by
> default for a long time.

I was going to say, it's currently not enabled by default, but it looks
like it gets enabled automagically when you load code from the old
cl-defstruct:

    (defun cl-struct-define (name docstring parent type named slots children-sym
                                  tag print)
      (unless type
        ;; Legacy defstruct, using tagged vectors.  Enable backward compatibility.
        (cl-old-struct-compat-mode 1))

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Thomas Fitzsimmons]

(CCing Alex Harsanyi.)

Noam Postavsky <npostavs@gmail.com> writes:

> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
>
>> Noam Postavsky <npostavs@gmail.com> writes:
>>
>>> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
>>>
>>>> (defun soap-resolve-references (element wsdl)
>>>
>>>>   (let ((resolver (get (aref element 0) 'soap-resolve-references)))
>>>
>>> Ah, then it should work to replace the (aref element 0) with (type-of
>>> element) as long as cl-old-struct-compat-mode is enabled.
>>
>> Yes, I retested with that change and it does work.  I guess I should
>> submit a patch to change all those occurrences in soap-client.
>
> I just noticed soap-client is also an ELPA package; then there is the
> additional complication that type-of will give the wrong answer in Emacs
> versions 25 and older.  So you would need something like
>
>     (let ((type (type-of element)))
>       (if (eq type 'vector)
>           (aref element 0) ; For Emacs 25 and earlier.
>         type))

OK, I had that on my list to investigate before patching soap-client,
the compatibility range of type-of, but you've done it for me, thanks.
A soap-type-of compatibility function probably makes sense to replace
the (aref ... 0) occurrences in soap-client and soap-inspect.

>> I'm hoping cl-old-struct-compat-mode will continue to be enabled by
>> default for a long time.
>
> I was going to say, it's currently not enabled by default, but it looks
> like it gets enabled automagically when you load code from the old
> cl-defstruct:
>
>     (defun cl-struct-define (name docstring parent type named slots children-sym
>                                   tag print)
>       (unless type
>         ;; Legacy defstruct, using tagged vectors.  Enable backward compatibility.
>         (cl-old-struct-compat-mode 1))

Ah, great, so that logic can stay in Emacs indefinitely without
negatively affecting sessions that don't load old byte-code.

Thomas

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Alex Harsanyi]

On Fri, Jun 8, 2018 at 10:18 PM, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
> (CCing Alex Harsanyi.)
>
> Noam Postavsky <npostavs@gmail.com> writes:
>
>> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
>>
>>> Noam Postavsky <npostavs@gmail.com> writes:
>>>
>>>> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
>>>>
>>>>> (defun soap-resolve-references (element wsdl)
>>>>
>>>>>   (let ((resolver (get (aref element 0) 'soap-resolve-references)))
>>>>
>>>> Ah, then it should work to replace the (aref element 0) with (type-of
>>>> element) as long as cl-old-struct-compat-mode is enabled.
>>>
>>> Yes, I retested with that change and it does work.  I guess I should
>>> submit a patch to change all those occurrences in soap-client.
>>
>> I just noticed soap-client is also an ELPA package; then there is the
>> additional complication that type-of will give the wrong answer in Emacs
>> versions 25 and older.  So you would need something like
>>
>>     (let ((type (type-of element)))
>>       (if (eq type 'vector)
>>           (aref element 0) ; For Emacs 25 and earlier.
>>         type))
>
> OK, I had that on my list to investigate before patching soap-client,
> the compatibility range of type-of, but you've done it for me, thanks.
> A soap-type-of compatibility function probably makes sense to replace
> the (aref ... 0) occurrences in soap-client and soap-inspect.
>
>>> I'm hoping cl-old-struct-compat-mode will continue to be enabled by
>>> default for a long time.
>>

The `(aref element 0)` trick is the only one that `soap-client` uses.  All
other struct usage is via accessor functions, the rest of the code does not
assume they are vectors.  It seems to me that `cl-old-struct-compat-mode` is
not needed.

Thomas, will you prepare the patch for soap-client?

Thanks,
Alex.

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Thomas Fitzsimmons]

[-- Attachment #1: Type: text/plain, Size: 2158 bytes --]

Alex Harsanyi <alexharsanyi@gmail.com> writes:

> On Fri, Jun 8, 2018 at 10:18 PM, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
>> (CCing Alex Harsanyi.)
>>
>> Noam Postavsky <npostavs@gmail.com> writes:
>>
>>> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
>>>
>>>> Noam Postavsky <npostavs@gmail.com> writes:
>>>>
>>>>> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
>>>>>
>>>>>> (defun soap-resolve-references (element wsdl)
>>>>>
>>>>>>   (let ((resolver (get (aref element 0) 'soap-resolve-references)))
>>>>>
>>>>> Ah, then it should work to replace the (aref element 0) with (type-of
>>>>> element) as long as cl-old-struct-compat-mode is enabled.
>>>>
>>>> Yes, I retested with that change and it does work.  I guess I should
>>>> submit a patch to change all those occurrences in soap-client.
>>>
>>> I just noticed soap-client is also an ELPA package; then there is the
>>> additional complication that type-of will give the wrong answer in Emacs
>>> versions 25 and older.  So you would need something like
>>>
>>>     (let ((type (type-of element)))
>>>       (if (eq type 'vector)
>>>           (aref element 0) ; For Emacs 25 and earlier.
>>>         type))
>>
>> OK, I had that on my list to investigate before patching soap-client,
>> the compatibility range of type-of, but you've done it for me, thanks.
>> A soap-type-of compatibility function probably makes sense to replace
>> the (aref ... 0) occurrences in soap-client and soap-inspect.
>>
>>>> I'm hoping cl-old-struct-compat-mode will continue to be enabled by
>>>> default for a long time.
>>>
>
> The `(aref element 0)` trick is the only one that `soap-client` uses.  All
> other struct usage is via accessor functions, the rest of the code does not
> assume they are vectors.  It seems to me that `cl-old-struct-compat-mode` is
> not needed.

OK.

> Thomas, will you prepare the patch for soap-client?

How about the attached?  I tested it in 26.1 with 25.1 excorporate.elc
byte-code, as well as with our test suite against 24.1, 25.1 and 26.1,
so it should be OK for GNU ELPA as well.

If this looks OK, I'll bump the soap-client.el version to 3.1.4 when I
push.

Thomas


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-soap-client-Add-byte-code-compatibility-function.patch --]
[-- Type: text/x-patch, Size: 11641 bytes --]

From 1638f42e494334696c3d3dc87c25af664086acaf Mon Sep 17 00:00:00 2001
From: Thomas Fitzsimmons <fitzsim@fitzsim.org>
Date: Fri, 8 Jun 2018 22:41:28 -0400
Subject: [PATCH] soap-client: Add byte-code compatibility function

* lisp/net/soap-client.el (soap-type-of): New function.
(soap-resolve-references, soap-decode-type)
(soap-encode-attributes, soap-encode-value): Replace aref
calls with calls to soap-type-of.
* lisp/net/soap-inspect.el (soap-sample-value, soap-inspect):
Replace aref calls with calls to soap-type-of.

Co-authored-by: Noam Postavsky <npostavs@gmail.com>
---
 lisp/net/soap-client.el  | 44 ++++++++++++++++++++++++++------------------
 lisp/net/soap-inspect.el | 44 ++++++++++++++++++++++----------------------
 2 files changed, 48 insertions(+), 40 deletions(-)

diff --git a/lisp/net/soap-client.el b/lisp/net/soap-client.el
index 3996da0..2535c34 100644
--- a/lisp/net/soap-client.el
+++ b/lisp/net/soap-client.el
@@ -685,8 +685,15 @@ soap-decode-xs-basic-type
         (anyType (soap-decode-any-type node))
         (Array (soap-decode-array node))))))
 
+(defun soap-type-of (element)
+  "Return the type of ELEMENT."
+  (let ((type (type-of element)))
+    (if (eq type 'vector)
+        (aref element 0) ; For Emacs 25 and earlier.
+      type)))
+
 ;; Register methods for `soap-xs-basic-type'
-(let ((tag (aref (make-soap-xs-basic-type) 0)))
+(let ((tag (soap-type-of (make-soap-xs-basic-type))))
   (put tag 'soap-attribute-encoder #'soap-encode-xs-basic-type-attributes)
   (put tag 'soap-encoder #'soap-encode-xs-basic-type)
   (put tag 'soap-decoder #'soap-decode-xs-basic-type))
@@ -915,7 +922,7 @@ soap-decode-xs-element
     (soap-decode-type type node)))
 
 ;; Register methods for `soap-xs-element'
-(let ((tag (aref (make-soap-xs-element) 0)))
+(let ((tag (soap-type-of (make-soap-xs-element))))
   (put tag 'soap-resolve-references #'soap-resolve-references-for-xs-element)
   (put tag 'soap-attribute-encoder #'soap-encode-xs-element-attributes)
   (put tag 'soap-encoder #'soap-encode-xs-element)
@@ -1011,7 +1018,7 @@ soap-resolve-references-for-xs-attribute
       (setf (soap-xs-attribute-reference attribute)
             (soap-wsdl-get reference wsdl predicate)))))
 
-(put (aref (make-soap-xs-attribute) 0)
+(put (soap-type-of (make-soap-xs-attribute))
      'soap-resolve-references #'soap-resolve-references-for-xs-attribute)
 
 (defun soap-resolve-references-for-xs-attribute-group (attribute-group wsdl)
@@ -1036,7 +1043,7 @@ soap-resolve-references-for-xs-attribute-group
         (setf (soap-xs-attribute-group-attribute-groups attribute-group)
               (soap-xs-attribute-group-attribute-groups resolved))))))
 
-(put (aref (make-soap-xs-attribute-group) 0)
+(put (soap-type-of (make-soap-xs-attribute-group))
      'soap-resolve-references #'soap-resolve-references-for-xs-attribute-group)
 
 ;;;;; soap-xs-simple-type
@@ -1374,7 +1381,7 @@ soap-decode-xs-simple-type
       (soap-validate-xs-simple-type value type))))
 
 ;; Register methods for `soap-xs-simple-type'
-(let ((tag (aref (make-soap-xs-simple-type) 0)))
+(let ((tag (soap-type-of (make-soap-xs-simple-type))))
   (put tag 'soap-resolve-references
        #'soap-resolve-references-for-xs-simple-type)
   (put tag 'soap-attribute-encoder #'soap-encode-xs-simple-type-attributes)
@@ -1927,7 +1934,7 @@ soap-decode-xs-complex-type
             (soap-xs-complex-type-indicator type)))))
 
 ;; Register methods for `soap-xs-complex-type'
-(let ((tag (aref (make-soap-xs-complex-type) 0)))
+(let ((tag (soap-type-of (make-soap-xs-complex-type))))
   (put tag 'soap-resolve-references
        #'soap-resolve-references-for-xs-complex-type)
   (put tag 'soap-attribute-encoder #'soap-encode-xs-complex-type-attributes)
@@ -2147,7 +2154,7 @@ soap-resolve-references
 function depending on the type of the ELEMENT.
 
 If ELEMENT has no resolver function, it is silently ignored."
-  (let ((resolver (get (aref element 0) 'soap-resolve-references)))
+  (let ((resolver (get (soap-type-of element) 'soap-resolve-references)))
     (when resolver
       (funcall resolver element wsdl))))
 
@@ -2272,13 +2279,13 @@ soap-resolve-references-for-port
 
 ;; Install resolvers for our types
 (progn
-  (put (aref (make-soap-message) 0) 'soap-resolve-references
+  (put (soap-type-of (make-soap-message)) 'soap-resolve-references
        'soap-resolve-references-for-message)
-  (put (aref (make-soap-operation) 0) 'soap-resolve-references
+  (put (soap-type-of (make-soap-operation)) 'soap-resolve-references
        'soap-resolve-references-for-operation)
-  (put (aref (make-soap-binding) 0) 'soap-resolve-references
+  (put (soap-type-of (make-soap-binding)) 'soap-resolve-references
        'soap-resolve-references-for-binding)
-  (put (aref (make-soap-port) 0) 'soap-resolve-references
+  (put (soap-type-of (make-soap-port)) 'soap-resolve-references
        'soap-resolve-references-for-port))
 
 (defun soap-wsdl-resolve-references (wsdl)
@@ -2685,16 +2692,17 @@ soap-decode-type
                (cond ((listp type)
                       (catch 'done
                         (dolist (union-member type)
-                          (let* ((decoder (get (aref union-member 0)
+                          (let* ((decoder (get (soap-type-of union-member)
                                                'soap-decoder))
                                  (result (ignore-errors
                                            (funcall decoder
                                                     union-member node))))
                             (when result (throw 'done result))))))
                      (t
-                      (let ((decoder (get (aref type 0) 'soap-decoder)))
+                      (let ((decoder (get (soap-type-of type) 'soap-decoder)))
                         (cl-assert decoder nil
-                                   "no soap-decoder for %s type" (aref type 0))
+                                   "no soap-decoder for %s type"
+                                   (soap-type-of type))
                         (funcall decoder type node))))))))))
 
 (defun soap-decode-any-type (node)
@@ -2878,9 +2886,9 @@ soap-encode-attributes
 
 Attributes are inserted in the current buffer at the current
 position."
-  (let ((attribute-encoder (get (aref type 0) 'soap-attribute-encoder)))
+  (let ((attribute-encoder (get (soap-type-of type) 'soap-attribute-encoder)))
     (cl-assert attribute-encoder nil
-               "no soap-attribute-encoder for %s type" (aref type 0))
+               "no soap-attribute-encoder for %s type" (soap-type-of type))
     (funcall attribute-encoder value type)))
 
 (defun soap-encode-value (value type)
@@ -2892,8 +2900,8 @@ soap-encode-value
 is to be encoded.  This is a generic function which finds an
 encoder function based on TYPE and calls that encoder to do the
 work."
-  (let ((encoder (get (aref type 0) 'soap-encoder)))
-    (cl-assert encoder nil "no soap-encoder for %s type" (aref type 0))
+  (let ((encoder (get (soap-type-of type) 'soap-encoder)))
+    (cl-assert encoder nil "no soap-encoder for %s type" (soap-type-of type))
     (funcall encoder value type))
   (when (soap-element-namespace-tag type)
     (add-to-list 'soap-encoded-namespaces (soap-element-namespace-tag type))))
diff --git a/lisp/net/soap-inspect.el b/lisp/net/soap-inspect.el
index 050be45..252b1f3 100644
--- a/lisp/net/soap-inspect.el
+++ b/lisp/net/soap-inspect.el
@@ -49,10 +49,10 @@ soap-sample-value
 
 This is a generic function, depending on TYPE a specific function
 will be called."
-  (let ((sample-value (get (aref type 0) 'soap-sample-value)))
+  (let ((sample-value (get (soap-type-of type) 'soap-sample-value)))
     (if sample-value
         (funcall sample-value type)
-      (error "Cannot provide sample value for type %s" (aref type 0)))))
+      (error "Cannot provide sample value for type %s" (soap-type-of type)))))
 
 (defun soap-sample-value-for-xs-basic-type (type)
   "Provide a sample value for TYPE, an xs-basic-type.
@@ -174,31 +174,31 @@ soap-sample-value-for-message
 
 (progn
   ;; Install soap-sample-value methods for our types
-  (put (aref (make-soap-xs-basic-type) 0)
+  (put (soap-type-of (make-soap-xs-basic-type))
        'soap-sample-value
        'soap-sample-value-for-xs-basic-type)
 
-  (put (aref (make-soap-xs-element) 0)
+  (put (soap-type-of (make-soap-xs-element))
        'soap-sample-value
        'soap-sample-value-for-xs-element)
 
-  (put (aref (make-soap-xs-attribute) 0)
+  (put (soap-type-of (make-soap-xs-attribute))
        'soap-sample-value
        'soap-sample-value-for-xs-attribute)
 
-  (put (aref (make-soap-xs-attribute) 0)
+  (put (soap-type-of (make-soap-xs-attribute))
        'soap-sample-value
        'soap-sample-value-for-xs-attribute-group)
 
-  (put (aref (make-soap-xs-simple-type) 0)
+  (put (soap-type-of (make-soap-xs-simple-type))
        'soap-sample-value
        'soap-sample-value-for-xs-simple-type)
 
-  (put (aref (make-soap-xs-complex-type) 0)
+  (put (soap-type-of (make-soap-xs-complex-type))
        'soap-sample-value
        'soap-sample-value-for-xs-complex-type)
 
-  (put (aref (make-soap-message) 0)
+  (put (soap-type-of (make-soap-message))
        'soap-sample-value
        'soap-sample-value-for-message))
 
@@ -222,7 +222,7 @@ soap-inspect
 The buffer is populated with information about ELEMENT with links
 to its sub elements.  If ELEMENT is the WSDL document itself, the
 entire WSDL can be inspected."
-  (let ((inspect (get (aref element 0) 'soap-inspect)))
+  (let ((inspect (get (soap-type-of element) 'soap-inspect)))
     (unless inspect
       (error "Soap-inspect: no inspector for element"))
 
@@ -507,39 +507,39 @@ soap-inspect-wsdl
 (progn
   ;; Install the soap-inspect methods for our types
 
-  (put (aref (make-soap-xs-basic-type) 0) 'soap-inspect
+  (put (soap-type-of (make-soap-xs-basic-type)) 'soap-inspect
        'soap-inspect-xs-basic-type)
 
-  (put (aref (make-soap-xs-element) 0) 'soap-inspect
+  (put (soap-type-of (make-soap-xs-element)) 'soap-inspect
        'soap-inspect-xs-element)
 
-  (put (aref (make-soap-xs-simple-type) 0) 'soap-inspect
+  (put (soap-type-of (make-soap-xs-simple-type)) 'soap-inspect
        'soap-inspect-xs-simple-type)
 
-  (put (aref (make-soap-xs-complex-type) 0) 'soap-inspect
+  (put (soap-type-of (make-soap-xs-complex-type)) 'soap-inspect
        'soap-inspect-xs-complex-type)
 
-  (put (aref (make-soap-xs-attribute) 0) 'soap-inspect
+  (put (soap-type-of (make-soap-xs-attribute)) 'soap-inspect
        'soap-inspect-xs-attribute)
 
-  (put (aref (make-soap-xs-attribute-group) 0) 'soap-inspect
+  (put (soap-type-of (make-soap-xs-attribute-group)) 'soap-inspect
        'soap-inspect-xs-attribute-group)
 
-  (put (aref (make-soap-message) 0) 'soap-inspect
+  (put (soap-type-of (make-soap-message)) 'soap-inspect
        'soap-inspect-message)
-  (put (aref (make-soap-operation) 0) 'soap-inspect
+  (put (soap-type-of (make-soap-operation)) 'soap-inspect
        'soap-inspect-operation)
 
-  (put (aref (make-soap-port-type) 0) 'soap-inspect
+  (put (soap-type-of (make-soap-port-type)) 'soap-inspect
        'soap-inspect-port-type)
 
-  (put (aref (make-soap-binding) 0) 'soap-inspect
+  (put (soap-type-of (make-soap-binding)) 'soap-inspect
        'soap-inspect-binding)
 
-  (put (aref (make-soap-port) 0) 'soap-inspect
+  (put (soap-type-of (make-soap-port)) 'soap-inspect
        'soap-inspect-port)
 
-  (put (aref (soap-make-wsdl "origin") 0) 'soap-inspect
+  (put (soap-type-of (soap-make-wsdl "origin")) 'soap-inspect
        'soap-inspect-wsdl))
 
 (provide 'soap-inspect)
-- 
1.8.3.1

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Alex Harsanyi]

On Sat, Jun 9, 2018 at 10:54 AM, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:

>
> How about the attached?  I tested it in 26.1 with 25.1 excorporate.elc
> byte-code, as well as with our test suite against 24.1, 25.1 and 26.1,
> so it should be OK for GNU ELPA as well.
>
> If this looks OK, I'll bump the soap-client.el version to 3.1.4 when I
> push.

Can you please add a comment referencing this bug to the definition of
`soap-type-of`.  Other than that, it looks good to me.

Alex.

>
> Thomas
>

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Thomas Fitzsimmons]

Alex Harsanyi <alexharsanyi@gmail.com> writes:

> On Sat, Jun 9, 2018 at 10:54 AM, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
>
>>
>> How about the attached?  I tested it in 26.1 with 25.1 excorporate.elc
>> byte-code, as well as with our test suite against 24.1, 25.1 and 26.1,
>> so it should be OK for GNU ELPA as well.
>>
>> If this looks OK, I'll bump the soap-client.el version to 3.1.4 when I
>> push.
>
> Can you please add a comment referencing this bug to the definition of
> `soap-type-of`.  Other than that, it looks good to me.

OK, will do.  I think I should put this on the emacs-26 branch, and also
on master so that it'll automatically get updated in GNU ELPA.

Thomas

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Thomas Fitzsimmons]

Alex Harsanyi <alexharsanyi@gmail.com> writes:

> On Sat, Jun 9, 2018 at 10:54 AM, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
>
>>
>> How about the attached?  I tested it in 26.1 with 25.1 excorporate.elc
>> byte-code, as well as with our test suite against 24.1, 25.1 and 26.1,
>> so it should be OK for GNU ELPA as well.
>>
>> If this looks OK, I'll bump the soap-client.el version to 3.1.4 when I
>> push.
>
> Can you please add a comment referencing this bug to the definition of
> `soap-type-of`.  Other than that, it looks good to me.

I added the comment, bumped the soap-client version to 3.1.4 and pushed
the patch to master so that GNU ELPA will be regenerated tonight.  I
also backported the patch to the emacs-26 branch since it fixes a
functional regression.

Thomas

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Eli Zaretskii]

> From: Thomas Fitzsimmons <fitzsim@fitzsim.org>
> Date: Mon, 11 Jun 2018 21:55:48 -0400
> Cc: Noam Postavsky <npostavs@gmail.com>, 31742-done@debbugs.gnu.org
> 
> I added the comment, bumped the soap-client version to 3.1.4 and pushed
> the patch to master so that GNU ELPA will be regenerated tonight.  I
> also backported the patch to the emacs-26 branch since it fixes a
> functional regression.

Thanks, but please in the future always ask before pushing stuff to
the release branch.  Not every regression fix should go to emacs-26,
only those which we consider "safe enough".

Also, if a change should be both on master and on emacs-26, push only
to the latter, and it will be later merged to master together with
other changes in emacs-26.  We only backport from master if initially
we didn't intend for a change to be on emacs-26; otherwise we merge in
the other direction.  (This is all explained in CONTRIBUTE under
"Branches".)

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Thomas Fitzsimmons]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Thomas Fitzsimmons <fitzsim@fitzsim.org>
>> Date: Mon, 11 Jun 2018 21:55:48 -0400
>> Cc: Noam Postavsky <npostavs@gmail.com>, 31742-done@debbugs.gnu.org
>> 
>> I added the comment, bumped the soap-client version to 3.1.4 and pushed
>> the patch to master so that GNU ELPA will be regenerated tonight.  I
>> also backported the patch to the emacs-26 branch since it fixes a
>> functional regression.
>
> Thanks, but please in the future always ask before pushing stuff to
> the release branch.  Not every regression fix should go to emacs-26,
> only those which we consider "safe enough".

OK, will do, sorry about that.

> Also, if a change should be both on master and on emacs-26, push only
> to the latter, and it will be later merged to master together with
> other changes in emacs-26.  We only backport from master if initially
> we didn't intend for a change to be on emacs-26; otherwise we merge in
> the other direction.  (This is all explained in CONTRIBUTE under
> "Branches".)

Yes, I knew about the preferred emacs-26 -> master merge direction, but
in this case I wanted to get the soap-client change out to GNU ELPA ASAP
so that users could unbreak their Emacs 25 byte-code.  A GNU ELPA
rebuild of soap-client requires a push to master.

I guess I should have undertaken a full gitmerge run myself after having
had the patch reviewed for emacs-26 on emacs-devel; I considered doing
so, but I thought the master -> emacs-26-backport approach would be less
risky.  But in the future I'll attempt the gitmerge route in cases like
this.

Thomas

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Alex Harsanyi]

On Tue, Jun 12, 2018 at 9:55 AM, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:

> I added the comment, bumped the soap-client version to 3.1.4 and pushed
> the patch to master so that GNU ELPA will be regenerated tonight.  I
> also backported the patch to the emacs-26 branch since it fixes a
> functional regression.

Hi Thomas,

Can you let me know where did you push the soap-client changes?

I can see that the package was released at gnu.elpa.org, but the
changes were not pushed either to the elpa.git repository at
https://git.savannah.gnu.org/git/emacs/elpa.git, or to my own copy of
it at https://github.com/alex-hhh/emacs-soap-client.

Thanks,
Alex.

>
> Thomas

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Thomas Fitzsimmons]

Alex Harsanyi <alexharsanyi@gmail.com> writes:

> On Tue, Jun 12, 2018 at 9:55 AM, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
>
>> I added the comment, bumped the soap-client version to 3.1.4 and pushed
>> the patch to master so that GNU ELPA will be regenerated tonight.  I
>> also backported the patch to the emacs-26 branch since it fixes a
>> functional regression.
>
> Hi Thomas,
>
> Can you let me know where did you push the soap-client changes?
>
> I can see that the package was released at gnu.elpa.org, but the
> changes were not pushed either to the elpa.git repository at
> https://git.savannah.gnu.org/git/emacs/elpa.git, or to my own copy of
> it at https://github.com/alex-hhh/emacs-soap-client.

I pushed it to the emacs.git master and emacs-26 branches:

http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=1feb2e221349f26ec26bc684e0cce2acecbed3ca

GNU ELPA pulls soap-client.el and soap-inspect.el from emacs.git's
master branch.

Let me know if you also want me to push the patch to
https://github.com/alex-hhh/emacs-soap-client.

Thomas

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Alex Harsanyi]

Hi Thomas,

I was not aware that the soap-client.el in the GNU Emacs source has been
updated -- the maintainers at the time wanted a version that just worked with
debbugs, and put the most recent version in ELPA.

There are two inconsistencies with the current system, which can cause some
confusion:

* the soap-client web page on elpa.gnu.org has a link to the elpa.git
  repository, but there is nothing there (the CGit and Gitweb links):
  http://elpa.gnu.org/packages/soap-client.html

* in the GNU Emacs source code, soap-client.el has a "Homepage" link that
  points back to my GitHub repository, which is now out of date.

I think we need to decide on where to keep the "official" development history
for this package.  As I see it, there are a few options:

* Keep it on GitHub, where it is now.  In this case, I think you should push
  the patch and the 3.1.4 tag there, and the ELPA links need to be fixed.

* Import it into elpa.git, as a separate branch (I see other packages are
  maintained like that).  In this case, the "Homepage" link should be updated
  to point to this source.

* Keep it in the GNU Emacs source tree (this will loose the git history, but
  maybe it is not important).  In this case the, both the "Homepage" link and
  the ELPA web page should be updated.

I have no strong preference for either option, apart for making it easier for
people to know where to go for the package source.  If the GitHub repository
is no longer the latest source code, I can archive it and point to the new
location.


Alex.

On Sun, Jun 17, 2018 at 8:36 AM, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
> Alex Harsanyi <alexharsanyi@gmail.com> writes:
>
>> On Tue, Jun 12, 2018 at 9:55 AM, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
>>
>>> I added the comment, bumped the soap-client version to 3.1.4 and pushed
>>> the patch to master so that GNU ELPA will be regenerated tonight.  I
>>> also backported the patch to the emacs-26 branch since it fixes a
>>> functional regression.
>>
>> Hi Thomas,
>>
>> Can you let me know where did you push the soap-client changes?
>>
>> I can see that the package was released at gnu.elpa.org, but the
>> changes were not pushed either to the elpa.git repository at
>> https://git.savannah.gnu.org/git/emacs/elpa.git, or to my own copy of
>> it at https://github.com/alex-hhh/emacs-soap-client.
>
> I pushed it to the emacs.git master and emacs-26 branches:
>
> http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=1feb2e221349f26ec26bc684e0cce2acecbed3ca
>
> GNU ELPA pulls soap-client.el and soap-inspect.el from emacs.git's
> master branch.
>
> Let me know if you also want me to push the patch to
> https://github.com/alex-hhh/emacs-soap-client.
>
> Thomas

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Noam Postavsky]

Alex Harsanyi <alexharsanyi@gmail.com> writes:

> * the soap-client web page on elpa.gnu.org has a link to the elpa.git
>   repository, but there is nothing there (the CGit and Gitweb links):
>   http://elpa.gnu.org/packages/soap-client.html

To clarify, the version served from elpa.gnu.org is from the emacs.git
repository.  See externals-list in the elpa.git repository:

;; List of packages that are maintained externally.
;; The list is made of elements of the form (NAME KIND URL OPTS...).
[...]
;; KIND can be one of:
[...]
;;  :core     = part of GNU Emacs repository.
[...]
("soap-client"		:core ("lisp/net/soap-client.el" "lisp/net/soap-inspect.el"))

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Thomas Fitzsimmons]

Hi Alex,

I pushed the patch and the 3.1.4 tag to
https://github.com/alex-hhh/emacs-soap-client.git so at least everything
is consistent again, except that the ELPA page doesn't point to
"Homepage".  I think that's a bug in GNU ELPA's generation scripts
though -- can you file a bug report for this?

As for repository organization, I'm fine with things as-is, I just have
to remember to push to GitHub anytime I push to emacs.git.

Right now the history is almost the same between emacs.git and
emacs-soap-client.git, and the files on the master branches are
identical.

Maybe in the future when elpa.git and emacs.git integration is better we
can use the elpa.git-separate-branch approach, and emacs.git will
automatically clone elpa.git as a submodule and everything will work in
core and for GNU ELPA updates.

In fact, soap-client is probably a good non-trivial package to
experiment with different Git organization approaches for such
core-and-GNU-ELPA packages.  There was a thread on emacs-devel about
this recently, in the context of adding BBDB to core.  I'm not working
on this Git integration effort directly right now, but I think some
people are from time to time.

Thomas

Alex Harsanyi <alexharsanyi@gmail.com> writes:

> Hi Thomas,
>
> I was not aware that the soap-client.el in the GNU Emacs source has been
> updated -- the maintainers at the time wanted a version that just worked with
> debbugs, and put the most recent version in ELPA.
>
> There are two inconsistencies with the current system, which can cause some
> confusion:
>
> * the soap-client web page on elpa.gnu.org has a link to the elpa.git
>   repository, but there is nothing there (the CGit and Gitweb links):
>   http://elpa.gnu.org/packages/soap-client.html
>
> * in the GNU Emacs source code, soap-client.el has a "Homepage" link that
>   points back to my GitHub repository, which is now out of date.
>
> I think we need to decide on where to keep the "official" development history
> for this package.  As I see it, there are a few options:
>
> * Keep it on GitHub, where it is now.  In this case, I think you should push
>   the patch and the 3.1.4 tag there, and the ELPA links need to be fixed.
>
> * Import it into elpa.git, as a separate branch (I see other packages are
>   maintained like that).  In this case, the "Homepage" link should be updated
>   to point to this source.
>
> * Keep it in the GNU Emacs source tree (this will loose the git history, but
>   maybe it is not important).  In this case the, both the "Homepage" link and
>   the ELPA web page should be updated.
>
> I have no strong preference for either option, apart for making it easier for
> people to know where to go for the package source.  If the GitHub repository
> is no longer the latest source code, I can archive it and point to the new
> location.
>
>
> Alex.
>
> On Sun, Jun 17, 2018 at 8:36 AM, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
>> Alex Harsanyi <alexharsanyi@gmail.com> writes:
>>
>>> On Tue, Jun 12, 2018 at 9:55 AM, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
>>>
>>>> I added the comment, bumped the soap-client version to 3.1.4 and pushed
>>>> the patch to master so that GNU ELPA will be regenerated tonight.  I
>>>> also backported the patch to the emacs-26 branch since it fixes a
>>>> functional regression.
>>>
>>> Hi Thomas,
>>>
>>> Can you let me know where did you push the soap-client changes?
>>>
>>> I can see that the package was released at gnu.elpa.org, but the
>>> changes were not pushed either to the elpa.git repository at
>>> https://git.savannah.gnu.org/git/emacs/elpa.git, or to my own copy of
>>> it at https://github.com/alex-hhh/emacs-soap-client.
>>
>> I pushed it to the emacs.git master and emacs-26 branches:
>>
>> http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=1feb2e221349f26ec26bc684e0cce2acecbed3ca
>>
>> GNU ELPA pulls soap-client.el and soap-inspect.el from emacs.git's
>> master branch.
>>
>> Let me know if you also want me to push the patch to
>> https://github.com/alex-hhh/emacs-soap-client.
>>
>> Thomas

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Thomas Fitzsimmons]

Alex Harsanyi <alexharsanyi@gmail.com> writes:

> On Fri, Jun 8, 2018 at 10:18 PM, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
>> (CCing Alex Harsanyi.)
>>
>> Noam Postavsky <npostavs@gmail.com> writes:
>>
>>> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
>>>
>>>> Noam Postavsky <npostavs@gmail.com> writes:
>>>>
>>>>> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
>>>>>
>>>>>> (defun soap-resolve-references (element wsdl)
>>>>>
>>>>>>   (let ((resolver (get (aref element 0) 'soap-resolve-references)))
>>>>>
>>>>> Ah, then it should work to replace the (aref element 0) with (type-of
>>>>> element) as long as cl-old-struct-compat-mode is enabled.
>>>>
>>>> Yes, I retested with that change and it does work.  I guess I should
>>>> submit a patch to change all those occurrences in soap-client.
>>>
>>> I just noticed soap-client is also an ELPA package; then there is the
>>> additional complication that type-of will give the wrong answer in Emacs
>>> versions 25 and older.  So you would need something like
>>>
>>>     (let ((type (type-of element)))
>>>       (if (eq type 'vector)
>>>           (aref element 0) ; For Emacs 25 and earlier.
>>>         type))
>>
>> OK, I had that on my list to investigate before patching soap-client,
>> the compatibility range of type-of, but you've done it for me, thanks.
>> A soap-type-of compatibility function probably makes sense to replace
>> the (aref ... 0) occurrences in soap-client and soap-inspect.
>>
>>>> I'm hoping cl-old-struct-compat-mode will continue to be enabled by
>>>> default for a long time.
>>>
>
> The `(aref element 0)` trick is the only one that `soap-client` uses.  All
> other struct usage is via accessor functions, the rest of the code does not
> assume they are vectors.  It seems to me that `cl-old-struct-compat-mode` is
> not needed.

I went back and tested with cl-old-struct-compat-mode disabled, with the
soap-type-of patch, and I get an earlier failure, via M-x excorporate:

Debugger entered--Lisp error: (wrong-type-argument soap-element [cl-struct-soap-port "ExchangeServicePort" [...]
  signal(wrong-type-argument (soap-element [cl-struct-soap-port "ExchangeServicePort" nil [...]
  (or (and (memq (type-of element) cl-struct-soap-element-tags) t) (signal 'wrong-type-argument (list 'soap-ele [...]
  (progn (or (and (memq (type-of element) cl-struct-soap-element-tags) t) (signal 'wrong-type-argument (list 's [...]
  (let ((name (progn (or (and (memq (type-of element) cl-struct-soap-element-tags) t) (signal 'wrong-type-argum [...]
  soap-namespace-put([cl-struct-soap-port "ExchangeServicePort" nil "https://[redacted]/ews/exchange.asmx" [...]
  exco--bind-wsdl(#s(soap-wsdl :origin "https://[redacted]/ews/exchange.asmx" :current-file nil :xmlschema- [...]
  #f(compiled-function (fsm state-data) #<bytecode 0x105c08f9>)(fsm-exco--fsm-47 (:identifier ("fitzsim@cisco. [...]
  fsm-update(fsm-exco--fsm-47 :retrieving-data (:identifier ("fitzsim@cisco.com" . "https://[redacted]/ews [...]
  fsm-send-sync(fsm-exco--fsm-47 t nil)
  apply(fsm-send-sync (fsm-exco--fsm-47 t nil))
  timer-event-handler([t 23323 41935 594437 nil fsm-send-sync (fsm-exco--fsm-47 t nil) nil 155000])

The old/new cl-struct mixing happens because excorporate.elc (25.1
byte-code) calls make-soap-port, and passes the result to soap-client
(26.1).  So at least for Excorporate, cl-old-struct-compat-mode is
needed (in addition to the (aref ... 0) -> (soap-type-of ...)
soap-client change).

Thomas

bug#31742: 26.1.50; excorporate.elc byte-compiled in Emacs 25.x fails in Emacs 26.1

[Noam Postavsky]

>> The `(aref element 0)` trick is the only one that `soap-client` uses.  All
>> other struct usage is via accessor functions, the rest of the code does not
>> assume they are vectors.  It seems to me that `cl-old-struct-compat-mode` is
>> not needed.

> I went back and tested with cl-old-struct-compat-mode disabled, with the
> soap-type-of patch, and I get an earlier failure, via M-x excorporate:

>  So at least for Excorporate, cl-old-struct-compat-mode is
> needed (in addition to the (aref ... 0) -> (soap-type-of ...)
> soap-client change).

Yeah, and for (type-of element) to properly substitute for (aref element
0) without cl-old-struct-compat-mode you would need

(defun soap-type-of (element)
  "Return the type of ELEMENT."
  (let ((type (type-of element)))
    (unless (eq type 'vector)           ; For Emacs 25 and earlier.
      (setq type (aref element 0)))
    (if (string-match "\\`cl-struct-" (symbol-name type)) ; Old-style struct.
        (intern (substring (symbol-name type) (match-end 0)))
      type)))

But since cl-old-struct-compat-mode activates automagically, may as well
take advantage of it.