wsdl test files and licensing

wsdl test files and licensing

[Michael Albinus]

Hi,

the soap-client people have written a test suite shich is obviously
useful. I would like to merge it into Emacs. This will take time,
because before this happens it shall use ert for running the tests. But
that's another story.

As test data, there are several *.wsdl files as well as soap request
examples (these are *.xml files). All of them are not licensed, except
Debbugs.wsdl taken from the GNU ELPA debbugs package.

Would it be necessary to bring all those files under GPLv3? As said, I
regard them as test data.

Best regards, Michael.

Re: wsdl test files and licensing

[David Kastrup]

Michael Albinus <michael.albinus@gmx.de> writes:

> the soap-client people have written a test suite shich is obviously
> useful. I would like to merge it into Emacs. This will take time,
> because before this happens it shall use ert for running the
> tests. But that's another story.
>
> As test data, there are several *.wsdl files as well as soap request
> examples (these are *.xml files). All of them are not licensed, except
> Debbugs.wsdl taken from the GNU ELPA debbugs package.

What does "are not licensed" mean?  If it means "there is no license to
distribute them", obviously we cannot distribute them.  That's a
no-brainer.

> Would it be necessary to bring all those files under GPLv3? As said, I
> regard them as test data.

If we distribute them as part of Emacs, the standard threshold is
"copyright assignment to FSF".  The FSF will then generally distribute
under GPLv3 or later, but that may change at some time subject to the
constraints spelled out in the copyright assignment contract.

Outside of a copyright assignment, stuff becomes more complex and
requires separate ok/go-ahead from Richard and/or the FSF copyright
clerk.  Off the cuff I'll be pretty sure that GPLv3 only won't cut it,
something like X11 license (or BSD without advertising clause) might
work, GPLv3+ might barely work.  But it's up to an individual decision
by Richard/copyright clerk whether a particular licensing of
non-assigned software can be accepted into the Emacs distribution.

For test data, it might be possible (if one cannot get the copyright
holders to sign a copyright assignment) to get a long with a copyright
disclaimer where the copyright holder will disclaim all copyright
interest in the respective files.  Again, this needs individual approval
by Richard/clerk.

-- 
David Kastrup

Re: wsdl test files and licensing

[Tassilo Horn]

David Kastrup <dak@gnu.org> writes:

>> As test data, there are several *.wsdl files as well as soap request
>> examples (these are *.xml files). All of them are not licensed,
>> except Debbugs.wsdl taken from the GNU ELPA debbugs package.
>
> What does "are not licensed" mean?  If it means "there is no license
> to distribute them", obviously we cannot distribute them.  That's a
> no-brainer.

I guess it means that it's a wild collection of files from unknown
sources with no explicit license header.  You can also search for WSDL
files.

  http://www.webservicex.net/globalweather.asmx?WSDL
  http://webservices.amazon.com/AWSECommerceService/AWSECommerceService.wsdl

Both don't declare any license.

> For test data, it might be possible (if one cannot get the copyright
> holders to sign a copyright assignment) to get a long with a copyright
> disclaimer where the copyright holder will disclaim all copyright
> interest in the respective files.  Again, this needs individual
> approval by Richard/clerk.

Would it help not to distribute such test data with emacs itself but to
have a script or the test itself download it if it's not already there?
I mean, those WSDL files describe (commercial) web services, so if I
wanted to use those services I'd need to download the files and analyze
them anyway.  Thus downloading them for running tests seems to be a
legit thing to do.

Bye,
Tassilo

Re: wsdl test files and licensing

[Alex Harsanyi]

[-- Attachment #1: Type: text/plain, Size: 1978 bytes --]

2014-11-12 20:47 GMT+08:00 David Kastrup <dak@gnu.org>:

> Michael Albinus <michael.albinus@gmx.de> writes:
>
> > the soap-client people have written a test suite shich is obviously
> > useful. I would like to merge it into Emacs. This will take time,
> > because before this happens it shall use ert for running the
> > tests. But that's another story.
> >
> > As test data, there are several *.wsdl files as well as soap request
> > examples (these are *.xml files). All of them are not licensed, except
> > Debbugs.wsdl taken from the GNU ELPA debbugs package.
>
> What does "are not licensed" mean?  If it means "there is no license to
> distribute them", obviously we cannot distribute them.  That's a
> no-brainer.
>

I will try to clarify what the situation is:

Each SOAP server provides a WSDL document which contains the definition of
the messages and their XML structure as accepted by the server.
soap-client.el (and other soap-clients) read this document and use it to
generate the XML requests and process the XML replies from the server.
While the WSDL itself is XML format, it is intended to be read by
computers, not humans (quite often, the documents are also generated
automatically from programing language interfaces).  These WSDL documents
are available to be downloaded from the SOAP servers themselves and are
necessary to be able to communicate with these servers.

We have a test suite which has about 11 WSDL documents from different
providers saved locally.  They are used to ensure that soap-client.el can
parse real-world WSDL documents.  None of these documents have any
copyright notice on them, except for Debbugs.wsdl and another one which has
an "All Rights Reserved" notice.

I wasn't sure if these documents can be distributed at all, so I kept the
test suite private, instead of adding it to the public repository which
hosts soap-client.el.  Michael suggested we ask for clarification on
emacs-devel, which he did :-)

Best Regards,
Alex.

[-- Attachment #2: Type: text/html, Size: 2585 bytes --]

Re: wsdl test files and licensing

[David Kastrup]

Alex Harsanyi <alexharsanyi@gmail.com> writes:

> We have a test suite which has about 11 WSDL documents from different
> providers saved locally.  They are used to ensure that soap-client.el
> can parse real-world WSDL documents.  None of these documents have any
> copyright notice on them, except for Debbugs.wsdl and another one
> which has an "All Rights Reserved" notice.

Well, it's obviously up to Richard and/or the FSF copyright clerk to
give any definite statement.  Basically, the "All Rights Reserved"
notice is redundant since that is the default by law.

It would be my guess that Richard would in this case be fine with a
"copyright disclaimer" as we are basically talking about samples rather
than maintained software: we don't get throown into a loop by ripping
any of them out: that can be done immediately without incurring
additional labor.

One would mail the legal departments/representatives of all known
sources with a polite request for giving such a disclaimer in order to
be able to include their WSDL documents in a test-suite which has the
side effect that their web sites will be one of several that are
guaranteed to work well.  One weeds out the refusals and looks for
suitable replacements, rinse and repeat.

Again: I am just second-guessing Richard here: anything definitive must
come from him or the copyright clerk.  It's just what I consider most
likely.

> I wasn't sure if these documents can be distributed at all, so I kept
> the test suite private, instead of adding it to the public repository
> which hosts soap-client.el.

That's perfectly smart, but it would be nice if we could get a public
test suite together.  And that will more likely than not requiring
mailing the respective people for disclaimers and/or recreating relevant
stuff from scratch when the respective permissions cannot be acquired.

-- 
David Kastrup

Re: wsdl test files and licensing

[Stefan Monnier]

> Would it be necessary to bring all those files under GPLv3?  As said, I
> regard them as test data.

To the extent that these are not really code files (similarly to he
test/indent/* files, which do contain code snippets but are not meant to
be runnable), I don't think it has to be exactly GPLv3+, but of course
it needs to have a license that lets us modify and distribute them.
And it probably needs the usual copyright assignment (unless, as usual,
it's small enough).

But you'd better ask Richard for a final decision on this.


        Stefan

Re: wsdl test files and licensing

[Michael Albinus]

David Kastrup <dak@gnu.org> writes:

>> I wasn't sure if these documents can be distributed at all, so I kept
>> the test suite private, instead of adding it to the public repository
>> which hosts soap-client.el.
>
> That's perfectly smart, but it would be nice if we could get a public
> test suite together.  And that will more likely than not requiring
> mailing the respective people for disclaimers and/or recreating relevant
> stuff from scratch when the respective permissions cannot be acquired.

Well, the situation is even a little bit more complicate. Most of the
SOAP servers offer their wsdl files at a well known url(*). I believe,
the soap-client test suite keeps the wsdl files locally for being able
to run offline tests, and also for consistency (results might change
when the wsdl changes).

In order to circumvent the license problem, one could run the tests
using wsdl files from their origin, downloading from the server. The
files wouldn't be distributed with Emacs. But one would loose the
advantages I've sketched above.

(*): One prominent exception are the debbugs servers. But Debbugs.wsdl
     used in the test suite is GPLv3 licensed :-)

Best regards, Michael.

Re: wsdl test files and licensing

[Stephen J. Turnbull]

Stefan Monnier writes:

 > > Would it be necessary to bring all those files under GPLv3?  As said, I
 > > regard them as test data.
 > 
 > To the extent that these are not really code files (similarly to he
 > test/indent/* files, which do contain code snippets but are not meant to
 > be runnable), I don't think it has to be exactly GPLv3+,

But I don't really think Richard or the FSF would be happy without a
copyright assignment in which case the current license is moot.

However, since these are test files, as a short-run workaround while
gathering "papers" or creating new tests, one possibility would be to
simply download the WSDL files during the test from the original sites
(assuming they're public).  If some source decides to change the file
and breaks the test, the maintainers would have a copy of the one that
they originally downloaded, discover the difference, and fix the
test.  The sites might not be happy to be used as test fodder, but
unless they're really unpopular I doubt they'd notice the load ;-).

Not really acceptable in the long run, but at least you could add the
code to Emacs and users could have some testing to give them
confidence things are working in the short run.

Steve

Re: wsdl test files and licensing

[David Kastrup]

Michael Albinus <michael.albinus@gmx.de> writes:

> David Kastrup <dak@gnu.org> writes:
>
>>> I wasn't sure if these documents can be distributed at all, so I kept
>>> the test suite private, instead of adding it to the public repository
>>> which hosts soap-client.el.
>>
>> That's perfectly smart, but it would be nice if we could get a public
>> test suite together.  And that will more likely than not requiring
>> mailing the respective people for disclaimers and/or recreating relevant
>> stuff from scratch when the respective permissions cannot be acquired.
>
> Well, the situation is even a little bit more complicate. Most of the
> SOAP servers offer their wsdl files at a well known url(*). I believe,
> the soap-client test suite keeps the wsdl files locally for being able
> to run offline tests, and also for consistency (results might change
> when the wsdl changes).
>
> In order to circumvent the license problem, one could run the tests
> using wsdl files from their origin, downloading from the server.

Test suites need to work offline, and they need to be reproducible.
I don't see that we get that easily with non-included material.


-- 
David Kastrup

Re: wsdl test files and licensing

[Michael Albinus]

David Kastrup <dak@gnu.org> writes:

> Test suites need to work offline, and they need to be reproducible.
> I don't see that we get that easily with non-included material.

The test suite (AFAIU, pls correct me otherwise, Alex) sends also SOAP
requests to the servers, and analyzes the results. Just wsdl parsing
tests could run offline, the other tests need a connection.

Of course, the test suite shall not be called by default when running
"make check" inside Emacs. It must be started explicitly.

Best regards, Michael.

Re: wsdl test files and licensing

[Alex Harsanyi]

[-- Attachment #1: Type: text/plain, Size: 999 bytes --]

2014-11-12 23:05 GMT+08:00 Michael Albinus <michael.albinus@gmx.de>:

> David Kastrup <dak@gnu.org> writes:
>
> > Test suites need to work offline, and they need to be reproducible.
> > I don't see that we get that easily with non-included material.
>
> The test suite (AFAIU, pls correct me otherwise, Alex) sends also SOAP
> requests to the servers, and analyzes the results. Just wsdl parsing
> tests could run offline, the other tests need a connection.
>

The entire test suite runs off-line.  It has three types of tests:

* loading offline WSDL files and checking that they were loaded correctly
* loading offline SOAP XML responses, decode them and compare them against
known (saved) responses
* generate SOAP XML requests and compare them against known (saved)
requests.

The XML responses are also based on real responses received from servers
and the XML requests are based on XML requests that we submitted to servers
and verified that the servers accepted them.  .

Best Regards,
Alex.

[-- Attachment #2: Type: text/html, Size: 1509 bytes --]

Re: wsdl test files and licensing

[Michael Albinus]

Alex Harsanyi <alexharsanyi@gmail.com> writes:

> The entire test suite runs off-line. It has three types of tests:
>
> * loading offline WSDL files and checking that they were loaded
> correctly
> * loading offline SOAP XML responses, decode them and compare them
> against known (saved) responses
> * generate SOAP XML requests and compare them against known (saved)
> requests.

I thought that at least some imports in the wsdl files must be resolved,
in order to generate proper request / compare responses. This requires
access to the respective servers.

Or do I miss something?

> Best Regards,
> Alex.

Best regards, Michael.