bug#3428: 23.0.94; Missing source files?

bug#3428: 23.0.94; Missing source files?

[Per Starbäck]

In GNU Emacs 23.0.94:

There are several files in emacs-23.0.94/lisp/international
that are automatically generated from files that aren't in the Emacs
distribution. Is that an oversight? At least

  charprop.el, eucjp-ms.el, uni-name.el and uni-old-name.el

have comments about being automatically generated from such files.
The files I'm missing are 

  unidata-gen.el, eucJP-13th.txt, eucJP-udc.txt, eucJP-ibmext.txt
  and UnicodeData.txt

judging by those comments.

bug#3428: 23.0.94; Missing source files?

[Chong Yidong]

> There are several files in emacs-23.0.94/lisp/international
> that are automatically generated from files that aren't in the Emacs
> distribution. Is that an oversight?

We intentionally don't include these source files.  They are not
small---UnicodeData.txt is 1.1MB---and they have no further use in
Emacs, so it doesn't make sense to include it in the Emacs distribution.
You only need to generate these files once, and that's it.

However, I think these files need to include the Unicode License
Agreement (a permissive free license).  I'll do that once Emacs CVS
comes back up.

bug#3428: 23.0.94; Missing source files?

[Richard Stallman]

    > There are several files in emacs-23.0.94/lisp/international
    > that are automatically generated from files that aren't in the Emacs
    > distribution. Is that an oversight?

    We intentionally don't include these source files.  They are not
    small---UnicodeData.txt is 1.1MB---and they have no further use in
    Emacs, so it doesn't make sense to include it in the Emacs distribution.
    You only need to generate these files once, and that's it.

We can package them separately if that is more convenient, but we are
ethically bound to make them available somehow.

Do we?

bug#3428: 23.0.94; Missing source files?

[Chong Yidong]

Richard Stallman <rms@gnu.org> writes:

>     We intentionally don't include these source files.  They are not
>     small---UnicodeData.txt is 1.1MB---and they have no further use in
>     Emacs, so it doesn't make sense to include it in the Emacs distribution.
>     You only need to generate these files once, and that's it.
>
> We can package them separately if that is more convenient, but we are
> ethically bound to make them available somehow.

Yes, in the admin/unidata directory of the Emacs source repository.

I propose to do the following, WDYT?

1. Add the following header to the generated files (which currently
   have no copyright header):

  ;; Copyright (C) 1991-2009 Unicode, Inc.
  ;; This file was generated from the Unicode data file at
  ;; http://www.unicode.org/Public/UNIDATA/UnicodeData.txt.
  ;; See lisp/international/README for the copyright and permission notice.

2. Add a README file to the lisp/international directory, with the
   following contents:

The following files in this directory are derived from the Unicode
Data File at http://www.unicode.org/Public/UNIDATA/UnicodeData.txt,
downloaded on 2006-05-23:

  charprop.el uni-bidi.el uni-category.el uni-combining.el
  uni-comment.el uni-decimal.el uni-decomposition.el uni-digit.el
  uni-lowercase.el uni-mirrored.el uni-name.el uni-numeric.el
  uni-old-name.el uni-titlecase.el uni-uppercase.el

These files were generated from UnicodeData.txt using unidata-gen.el,
which is not included in the Emacs distribution; it can be found in
the admin/unidata directory of the Emacs source repository.

The file UnicodeData.txt is used under the terms of the following
Copyright and Permission Notice:


Copyright (C) 1991-2009 Unicode, Inc. All rights reserved. Distributed
under the Terms of Use in http://www.unicode.org/copyright.html.

Permission is hereby granted, free of charge, to any person obtaining
a copy of the Unicode data files and any associated documentation (the
"Data Files") or Unicode software and any associated documentation
(the "Software") to deal in the Data Files or Software without
restriction, including without limitation the rights to use, copy,
modify, merge, publish, distribute, and/or sell copies of the Data
Files or Software, and to permit persons to whom the Data Files or
Software are furnished to do so, provided that (a) the above copyright
notice(s) and this permission notice appear with all copies of the
Data Files or Software, (b) both the above copyright notice(s) and
this permission notice appear in associated documentation, and (c)
there is clear notice in each modified Data File or in the Software as
well as in the documentation associated with the Data File(s) or
Software that the data or software has been modified.

THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT OF THIRD PARTY RIGHTS. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR HOLDERS INCLUDED IN THIS NOTICE BE LIABLE FOR ANY CLAIM, OR
ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE DATA FILES OR
SOFTWARE.

Except as contained in this notice, the name of a copyright holder
shall not be used in advertising or otherwise to promote the sale, use
or other dealings in these Data Files or Software without prior
written authorization of the copyright holder.

bug#3428: 23.0.94; Missing source files?

[Richard Stallman]

This looks good, but this

    which is not included in the Emacs distribution; it can be found in
    the admin/unidata directory of the Emacs source repository.

should say precisely WHERE the Emacs source repository is.