Proposed gnulib renames

Proposed gnulib renames

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 4185 bytes --]

[intentionally breaking threading and retitling this, to try and make it
easier to see replies to just this aspect of the thread]

[well, my first attempt at breaking threading failed; apologies for the
duplicate, but here's a resend with identical contents to make good on
the promise of a new thread]

On 01/25/2011 11:01 PM, Eli Zaretskii wrote:
> > Here's a compromise that at least I can live with; hope others can,
> > too:
> >
> >  1) For c++defs.h and the lib/*.in.h files: rely on the automatic
> >     renaming by djtar.  Files that reference those will be edited by
> >     config.bat as part of configuring Emacs for the MS-DOS build.

Paul already suggested renaming c++defs.h to cxxdefs.h in gnulib, which
makes sense to me in light of POSIX restrictions on portable filenames;
however, this module belongs to Bruno, so it is his call.

As for the *.in.h files, the ONLY other files that refer to those at
build time are the Makefile snippets in module files that convert them
over to *.h replacement headers.  Here's a link to some of the
back-discussion where we first settled on the name *.in.h in the first
place in Oct 2007 (we were previously using *_.h, but underscores are
painful to type in daily use; and also on the table at the time was the
rejected idea of *.h.in and *.hin):

http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/11185/focus=11206

and given that the renaming from *_.h -> *.in.h was practically
mechanical, a conversion from *.in.h -> *-in.h would likewise be
mechanical, but I'm not sure whether to make that jump yet:

http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/11182/focus=11352

that thread also included a quote from Eli, predicting the death of an
emacs DOS build (for good or for bad, that prediction has not come true):

http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/11182/focus=11234

> >
> >  2) For the 3 m4/gnulib-*.m4 files: rename them.  I suggested one way
> >     of renaming, but there's nothing sacred about that; any renaming
> >     that will get rid of the name clash is fine with me.

This would involve a change mainly to gnulib-tool (2 of the 3 gnulib-c*
files are generated; there's also gnulib-tool.m4 to avoid clashing
with).  Changing gnulib-cache.m4 would have downstream effects on all
gnulib clients; it could be done with a NEWS entry, but I'd rather avoid
that.  But the other two files (gnulib-common.m4 and gnulib-comp.m4)
seem like fair game; how about the names gnulib-prereq.m4 (since all the
common code is prerequisite to the rest of gnulib) and gnulib-list.m4
(since comp contains the computed list of files/macros installed by
gnulib-tool).

> > As long as the list of files that get handled by 1) is relatively
> > small and kept under control, this is manageable.

Right now, there is the c++defs naming issue (with user-visible changes
to all gnulib clients, but worth making).

Also, there are currently 63 *.in.h files available in gnulib (I'm not
sure how many will ever be used by emacs), but I'd rather see the same
change made to all 63 files (and their corresponding modules) at once if
the change is made upstream in gnulib.  I agree that use of gnulib-tool
--local-dir won't quite work; it could easily patch the module
descriptions (and makefile snippets) to refer to *-in.h files with
minimal risk of too many upstream changes to track, but the rename
itself is not possible via simple patch(1) files (yes, GNU patchutils is
adding support for git-style rename patches, which would be much easier
to maintain, but I don't know if we should rely on that yet).  But if
gnulib makes no change to these 63 files, then that's an upper bound on
the possible complexity of Eli hand-maintaining the rename within the
DOS build of emacs.

> > I hope that as long as the list of files that are handled by 2) is
> > short (3 for now), that would be regarded as manageable and acceptable
> > by gnulib people.

It seems reasonable to me, but I'm not the primary author of
gnulib-tool.  Bruno?

-- 
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: Proposed gnulib renames

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 2820 bytes --]

On 01/26/2011 08:58 AM, Eli Zaretskii wrote:
>> Paul already suggested renaming c++defs.h to cxxdefs.h in gnulib, which
>> makes sense to me in light of POSIX restrictions on portable filenames;
>> however, this module belongs to Bruno, so it is his call.
>
> And Bruno already voiced his objections, so I guess I will have to
> live will that.  It's not a big deal to edit with Sed the few files
> that refer to c++defs.h, as part of the config.bat run.  Of course, if
> Bruno will change his mind, it will be even better.

Ah - http://lists.gnu.org/archive/html/bug-gnulib/2011-01/msg00323.html

But Bruno's suggestion of using gnulib-tool --local-dir for the
emacs-only rename for c++defs might make sense; this module is less
volatile than the *.in.h issue, and so is less likely to cause perpetual
merge grief in maintaining such a patch.

>
>> and given that the renaming from *_.h -> *.in.h was practically
>> mechanical, a conversion from *.in.h -> *-in.h would likewise be
>> mechanical, but I'm not sure whether to make that jump yet:
>>
>> http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/11182/focus=11352
>
> I'm okay with renaming them to use underscores.  But if gnulib
> maintainers object, the djtar utility will take care of these, too.

The whole point is that they used to be underscore, but we switched as
part of a process of preferring dash over underscore.  If we do the
rename in gnulib, I'd prefer it to be to a dash.

> Editing Makefile.in to refer to them in config.h should not be a big
> issue.

Good to hear - if the tar file automatically flattens them into a
particular name, and your script can touch up the Makefile.in to refer
to that flattened name, then we've isolated the problem outside of gnulib.

>> This would involve a change mainly to gnulib-tool (2 of the 3 gnulib-c*
>> files are generated; there's also gnulib-tool.m4 to avoid clashing
>> with).  Changing gnulib-cache.m4 would have downstream effects on all
>> gnulib clients; it could be done with a NEWS entry, but I'd rather avoid
>> that.  But the other two files (gnulib-common.m4 and gnulib-comp.m4)
>> seem like fair game; how about the names gnulib-prereq.m4 (since all the
>> common code is prerequisite to the rest of gnulib) and gnulib-list.m4
>> (since comp contains the computed list of files/macros installed by
>> gnulib-tool).
>
> Of course, it is enough to rename only 2 of the 3, to avoid file-name
> clashes.  So if your suggestion to rename gnulib-common.m4 and
> gnulib-comp.m4 is accepted, that would solve the problem entirely.

Bruno has not yet weighed in on this proposal (his earlier objection was
just to the c++defs naming).

-- 
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: Proposed gnulib renames

[Jim Meyering]

Eric Blake wrote:
> [intentionally breaking threading and retitling this, to try and make it
> easier to see replies to just this aspect of the thread]
>
> [well, my first attempt at breaking threading failed; apologies for the
> duplicate, but here's a resend with identical contents to make good on
> the promise of a new thread]
>
> On 01/25/2011 11:01 PM, Eli Zaretskii wrote:
>> > Here's a compromise that at least I can live with; hope others can,
>> > too:
>> >
>> >  1) For c++defs.h and the lib/*.in.h files: rely on the automatic
>> >     renaming by djtar.  Files that reference those will be edited by
>> >     config.bat as part of configuring Emacs for the MS-DOS build.
>
> Paul already suggested renaming c++defs.h to cxxdefs.h in gnulib, which
> makes sense to me in light of POSIX restrictions on portable filenames;
> however, this module belongs to Bruno, so it is his call.
>
> As for the *.in.h files, the ONLY other files that refer to those at
> build time are the Makefile snippets in module files that convert them
> over to *.h replacement headers.  Here's a link to some of the
> back-discussion where we first settled on the name *.in.h in the first
> place in Oct 2007 (we were previously using *_.h, but underscores are
> painful to type in daily use; and also on the table at the time was the
> rejected idea of *.h.in and *.hin):

Hi Eric,

While names containing two '.'s constitute part of the problem,
(I have no fundamental objection to renaming those, btw)
there's a more insidious one.

Here are some of the reasons to try hard to avoid 8.3 constraints
altogether.  The following are some of the tuples of names in gnulib
that collide in 8.3-land:

  basename-lgpl.c lib/basename-lgpl.c
  basename.c lib/basename.c

  c-strcasecmp.c lib/c-strcasecmp.c
  c-strcasestr.c lib/c-strcasestr.c
  c-strncasecmp.c lib/c-strncasecmp.c

  c-strcaseeq.h lib/c-strcaseeq.h
  c-strcasestr.h lib/c-strcasestr.h

  canonicalize-lgpl.c lib/canonicalize-lgpl.c
  canonicalize.c lib/canonicalize.c

  ctype_alnum.c lib/unictype/ctype_alnum.c
  ctype_alpha.c lib/unictype/ctype_alpha.c

  ctype_alnum.h lib/unictype/ctype_alnum.h
  ctype_alpha.h lib/unictype/ctype_alpha.h

  dup-safer-flag.c lib/dup-safer-flag.c
  dup-safer.c lib/dup-safer.c

  filenamecat-lgpl.c lib/filenamecat-lgpl.c
  filenamecat.c lib/filenamecat.c

  fd-safer-flag.c lib/fd-safer-flag.c
  fd-safer.c lib/fd-safer.c

  decompose-internal.c lib/uninorm/decompose-internal.c
  decomposing-form.c lib/uninorm/decomposing-form.c
  decomposition-table.c lib/uninorm/decomposition-table.c
  decomposition.c lib/uninorm/decomposition.c

  mbmemcasecmp.c lib/mbmemcasecmp.c
  mbmemcasecoll.c lib/mbmemcasecoll.c

  pipe-filter-gi.c lib/pipe-filter-gi.c
  pipe-filter-ii.c lib/pipe-filter-ii.c

  pr_bidi_arabic_digit.c lib/unictype/pr_bidi_arabic_digit.c
  pr_bidi_arabic_right_to_left.c lib/unictype/pr_bidi_arabic_right_to_left.c
  pr_bidi_block_separator.c lib/unictype/pr_bidi_block_separator.c
  pr_bidi_boundary_neutral.c lib/unictype/pr_bidi_boundary_neutral.c
  pr_bidi_common_separator.c lib/unictype/pr_bidi_common_separator.c
  pr_bidi_control.c lib/unictype/pr_bidi_control.c
  pr_bidi_embedding_or_override.c lib/unictype/pr_bidi_embedding_or_override.c
  pr_bidi_eur_num_separator.c lib/unictype/pr_bidi_eur_num_separator.c
  pr_bidi_eur_num_terminator.c lib/unictype/pr_bidi_eur_num_terminator.c
  pr_bidi_european_digit.c lib/unictype/pr_bidi_european_digit.c
  pr_bidi_hebrew_right_to_left.c lib/unictype/pr_bidi_hebrew_right_to_left.c
  pr_bidi_left_to_right.c lib/unictype/pr_bidi_left_to_right.c
  pr_bidi_non_spacing_mark.c lib/unictype/pr_bidi_non_spacing_mark.c
  pr_bidi_other_neutral.c lib/unictype/pr_bidi_other_neutral.c
  pr_bidi_pdf.c lib/unictype/pr_bidi_pdf.c
  pr_bidi_segment_separator.c lib/unictype/pr_bidi_segment_separator.c
  pr_bidi_whitespace.c lib/unictype/pr_bidi_whitespace.c

  readlink.c lib/readlink.c
  readlinkat.c lib/readlinkat.c

  readtokens.c lib/readtokens.c
  readtokens0.c lib/readtokens0.c

  spawn_faction_addclose.c lib/spawn_faction_addclose.c
  spawn_faction_adddup2.c lib/spawn_faction_adddup2.c
  spawn_faction_addopen.c lib/spawn_faction_addopen.c
  spawn_faction_destroy.c lib/spawn_faction_destroy.c
  spawn_faction_init.c lib/spawn_faction_init.c
  spawnattr_destroy.c lib/spawnattr_destroy.c
  spawnattr_getdefault.c lib/spawnattr_getdefault.c
  spawnattr_getflags.c lib/spawnattr_getflags.c
  spawnattr_getpgroup.c lib/spawnattr_getpgroup.c
  spawnattr_getschedparam.c lib/spawnattr_getschedparam.c
  spawnattr_getschedpolicy.c lib/spawnattr_getschedpolicy.c
  spawnattr_getsigmask.c lib/spawnattr_getsigmask.c
  spawnattr_init.c lib/spawnattr_init.c
  spawnattr_setdefault.c lib/spawnattr_setdefault.c
  spawnattr_setflags.c lib/spawnattr_setflags.c
  spawnattr_setpgroup.c lib/spawnattr_setpgroup.c
  spawnattr_setschedparam.c lib/spawnattr_setschedparam.c
  spawnattr_setschedpolicy.c lib/spawnattr_setschedpolicy.c
  spawnattr_setsigmask.c lib/spawnattr_setsigmask.c

  strerror.c lib/strerror.c
  strerror_r.c lib/strerror_r.c

  striconv.c lib/striconv.c
  striconveh.c lib/striconveh.c
  striconveha.c lib/striconveha.c

  u16-casecmp.c lib/unicase/u16-casecmp.c
  u16-casecoll.c lib/unicase/u16-casecoll.c
  u16-casefold.c lib/unicase/u16-casefold.c
  u16-casemap.c lib/unicase/u16-casemap.c

  xstrtoul.c lib/xstrtoul.c
  xstrtoull.c lib/xstrtoull.c

  ...

Perhaps no pair is used by emacs at the moment, but
if we're going to accommodate by renaming every .in.h file,
eventually we will have to address the other sort of problem, too.

Renaming files like those to avoid the 8.3 collisions does not seem
like the right move, especially since we (at least I) have no idea of
the size of the user base we would be accommodating.  Are we even sure
that it is necessary?  I would feel better about this if Eli were gung-ho
to use something like doslfn, had tried it, but had found that there
were some fundamentally unfixable and show-stopping flaw in its design.

Re: Proposed gnulib renames

[Eli Zaretskii]

> From: Jim Meyering <jim@meyering.net>
> Cc: Eli Zaretskii <eliz@gnu.org>,  Paul Eggert <eggert@CS.UCLA.EDU>,  bug-gnulib <bug-gnulib@gnu.org>,  cyd@stupidchicken.com,  emacs-devel@gnu.org,  monnier@IRO.UMontreal.CA,  Bruno Haible <bruno@clisp.org>
> Date: Wed, 26 Jan 2011 17:36:42 +0100
> 
> Here are some of the reasons to try hard to avoid 8.3 constraints
> altogether.  The following are some of the tuples of names in gnulib
> that collide in 8.3-land:

I have no doubt that there are a lot of these in gnulib.  But the
chances that Emacs will ever want to use them are slim at the moment.
I don't see any reasons to try to solve a theoretical problem which
might never come our way.  If it does come, and in these quantities,
not even I will dream of asking for such wholesale renames.  Or maybe
the DOS build will be dead by then, who knows?

So I suggest not to try to solve a problem until it is on the table.
I think what Bruno proposed is reasonable and easily maintainable, so
I would go for that.

> I would feel better about this if Eli were gung-ho
> to use something like doslfn, had tried it, but had found that there
> were some fundamentally unfixable and show-stopping flaw in its design.

Talking about some kind of add-on that all users of DOS Emacs will
have to install is a no-starter.  Whoever wants long file names in
DJGPP programs can much more easily install any version of Windows and
be done with it.  OTOH, those who do need DOS want it as plain and
un-tainted by unmaintained add-ons of dubious quality as possible.

Re: Proposed gnulib renames

[Jim Meyering]

Eli Zaretskii wrote:
>> From: Jim Meyering <jim@meyering.net>
>> Cc: Eli Zaretskii <eliz@gnu.org>, Paul Eggert <eggert@CS.UCLA.EDU>,
>> bug-gnulib <bug-gnulib@gnu.org>, cyd@stupidchicken.com,
>> emacs-devel@gnu.org, monnier@IRO.UMontreal.CA, Bruno Haible
>> <bruno@clisp.org>
>> Date: Wed, 26 Jan 2011 17:36:42 +0100
>>
>> Here are some of the reasons to try hard to avoid 8.3 constraints
>> altogether.  The following are some of the tuples of names in gnulib
>> that collide in 8.3-land:
>
> I have no doubt that there are a lot of these in gnulib.  But the
> chances that Emacs will ever want to use them are slim at the moment.
> I don't see any reasons to try to solve a theoretical problem which
> might never come our way.  If it does come, and in these quantities,
> not even I will dream of asking for such wholesale renames.  Or maybe
> the DOS build will be dead by then, who knows?
>
> So I suggest not to try to solve a problem until it is on the table.
> I think what Bruno proposed is reasonable and easily maintainable, so
> I would go for that.
>
>> I would feel better about this if Eli were gung-ho
>> to use something like doslfn, had tried it, but had found that there
>> were some fundamentally unfixable and show-stopping flaw in its design.
>
> Talking about some kind of add-on that all users of DOS Emacs will
> have to install is a no-starter.  Whoever wants long file names in
> DJGPP programs can much more easily install any version of Windows and
> be done with it.  OTOH, those who do need DOS want it as plain and
> un-tainted by unmaintained add-ons of dubious quality as possible.

There are two issues here.
Requiring doslfn solely for those *building* emacs on DOS would
allow the names of all C source and other build-only files to be "long".
To gain that benefit, users would *not* have to install doslfn.

However, to extend that benefit to files loaded by emacs
at run time, they would.

Re: Proposed gnulib renames

[Eli Zaretskii]

> From: Jim Meyering <jim@meyering.net>
> Cc: eblake@redhat.com,  eggert@CS.UCLA.EDU,  bug-gnulib@gnu.org,  cyd@stupidchicken.com,  emacs-devel@gnu.org,  monnier@IRO.UMontreal.CA,  bruno@clisp.org
> Date: Wed, 26 Jan 2011 19:59:18 +0100
> 
> Requiring doslfn solely for those *building* emacs on DOS would
> allow the names of all C source and other build-only files to be "long".
> To gain that benefit, users would *not* have to install doslfn.
> 
> However, to extend that benefit to files loaded by emacs
> at run time, they would.

There's no reason to believe that file-name conflicts will be limited
to C files only.  C files in Emacs are added very infrequently, while
Lisp files much more frequently.

Re: Proposed gnulib renames

[Jim Meyering]

Eli Zaretskii wrote:
>> From: Jim Meyering <jim@meyering.net>
>> Cc: eblake@redhat.com, eggert@CS.UCLA.EDU, bug-gnulib@gnu.org,
>> cyd@stupidchicken.com, emacs-devel@gnu.org,
>> monnier@IRO.UMontreal.CA, bruno@clisp.org
>> Date: Wed, 26 Jan 2011 19:59:18 +0100
>>
>> Requiring doslfn solely for those *building* emacs on DOS would
>> allow the names of all C source and other build-only files to be "long".
>> To gain that benefit, users would *not* have to install doslfn.
>>
>> However, to extend that benefit to files loaded by emacs
>> at run time, they would.
>
> There's no reason to believe that file-name conflicts will be limited
> to C files only.

No one who has followed this thread could think that.
The point is that we do gain some benefit, even if we can
assume use of doslfn only for the build process.
Obviously, it would be better to be able to assume
a reasonable file system model for run-time, too.

> C files in Emacs are added very infrequently, while
> Lisp files much more frequently.

With gnulib, it is easy to add many (via dependencies),
at least initially.

Re: Proposed gnulib renames

[Paul Eggert]

On 01/26/11 08:36, Jim Meyering wrote:
> Renaming files like those to avoid the 8.3 collisions does not seem
> like the right move, especially since we (at least I) have no idea of
> the size of the user base we would be accommodating.

I agree that wholesale renaming is not a realistic option.

It appears that Eli has a solution for c++defs and there's
no need to worry about it right now.

The *.in.h problem is more serious, though, as it limits
include file names to 7 letters before the dot.  For example,
in DOS, the name "fnmatch-in.h" is equivalent to "fnmatch.h"
(the excess chars "-in" are ignored by the file system),
so one cannot have a Make rule that creates the latter from
the former.

Currently this is not a problem since all the .in.h files
that Emacs uses are 7 characters or less.  But this is not
a tenable restriction in the long run.

One thing does strike me, though, as being a useful thing we
can do regardless of the DOS business.  Currently we build
lib/c++defs.h from build-aux/c++defs.h as part of ordinary
'make'.  But lib/c++defs.h is the same on all platforms.
It would save work for ordinary 'make' if we distributed
lib/c++defs.h, so that the maintainer can build it rather than
the builder.  Similarly for lib/warn-on-use.h and lib/arg-nonnull.h.

I can propose gnulib changes along those lines, unless there's
some objection.  I don't expect this to affect the difficulty
of the Emacs MS-DOS build.

Re: Proposed gnulib renames

[Eli Zaretskii]

> Date: Wed, 26 Jan 2011 16:39:43 -0800
> From: Paul Eggert <eggert@cs.ucla.edu>
> CC: Eric Blake <eblake@redhat.com>, Eli Zaretskii <eliz@gnu.org>, 
>  bug-gnulib <bug-gnulib@gnu.org>,
>  cyd@stupidchicken.com, emacs-devel@gnu.org, monnier@IRO.UMontreal.CA, 
>  Bruno Haible <bruno@clisp.org>
> 
> The *.in.h problem is more serious, though, as it limits
> include file names to 7 letters before the dot.

8, not 7.

> For example,
> in DOS, the name "fnmatch-in.h" is equivalent to "fnmatch.h"
> (the excess chars "-in" are ignored by the file system),

No, they are not equivalent.  fnmatch-in.h is equivalent to
fnmatch-.h, which is different from fnmatch.h.

> Currently this is not a problem since all the .in.h files
> that Emacs uses are 7 characters or less.  But this is not
> a tenable restriction in the long run.

With my suggestion, djtar will extract verylongname.in.h as
verylong.in-, so I see no problem here.

Thanks.

Files from gnulib

[Eli Zaretskii]

Some of the new files added from gnulib use file names that are
invalid on 8+3 (aka MSDOS) filesystems.  These files are:

 c++defs.h -- the `+' character is not allowed in file names
 lib/stddef.in.h lib/time.in.h lib/unistd.in.h -- 2 dots are not allowed

In addition, several file names in the m4/ subdirectory clash after
8+3 truncation:

 gnulib-cache.m4  gnulib-common.m4  gnulib-comp.m4

This will cause problems when unpacking the emacs tarball on those
filesystems, something we managed to avoid until now.

Can these files be renamed, please?  I suggest cxxdefs.h for the first
and FOO.in for the FOO.in.h files.  For the files in m4/, how about
transposing the "gnulib" part with the rest, i.e. cache-gnulib.m4
etc.?

(I would remove the files myself, but I don't know what changes in the
gnulib imports will that require.)

TIA

Re: Files from gnulib

[Paul Eggert]

[Adding bug-gnulib to the thread that started in
 <http://lists.gnu.org/archive/html/emacs-devel/2011-01/msg00761.html>]:

On 01/23/2011 04:15 AM, Eli Zaretskii wrote:
> Some of the new files added from gnulib use file names that are
> invalid on 8+3 (aka MSDOS) filesystems.  These files are:
> 
>  c++defs.h -- the `+' character is not allowed in file names

I am sympathetic to the proposal to rename c++defs.h to cxxdefs.h,
as "+" is not in the POSIX portable file name character set; see
<http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_276>.

>  lib/stddef.in.h lib/time.in.h lib/unistd.in.h -- 2 dots are not allowed

The Emacs trunk already has seventeen other files
with 2 dots in their file names, with names like
lisp/gnus/.dir-locals.el and admin/charsets/mapfiles/symbol.txt.gz.
Since these haven't been a problem, why would file names
like getopt.in.h be a problem?

> In addition, several file names in the m4/ subdirectory clash after
> 8+3 truncation:
> 
>  gnulib-cache.m4  gnulib-common.m4  gnulib-comp.m4

Again, the Emacs trunk already has several instances of truncation
after 8+3 limits, such as lisp/org/org-compat.el versus
lisp/org/org-complete.el, and test/cedet/semantic-ia-utest.el
versus test/cedet/semantic-tests.el, and I don't see why
files imported from gnulib would be different.

> This will cause problems when unpacking the emacs tarball on those
> filesystems, something we managed to avoid until now.
> 
> Can these files be renamed, please?  I suggest cxxdefs.h for the first
> and FOO.in for the FOO.in.h files.  For the files in m4/, how about
> transposing the "gnulib" part with the rest, i.e. cache-gnulib.m4
> etc.?
> 
> (I would remove the files myself, but I don't know what changes in the
> gnulib imports will that require.)

Developers on Microsoft Windows platforms all have access to
better file systems these days, surely.  And as shown above
Emacs does not respect those limitations in other parts
of its source tree.  So, other than c++defs.h, I don't see
the need for renaming these files.

Re: Files from gnulib

[Stefan Monnier]

> The Emacs trunk already has seventeen other files
> with 2 dots in their file names, with names like
> lisp/gnus/.dir-locals.el

I don't know how MSDOS handles names that start with dot, so either this
is a special case that's OK, or it's indeed a problem which we hadn't
noticed yet.

> and admin/charsets/mapfiles/symbol.txt.gz.

This one, OTOH is OK because the `admin' subdir is not included in the
tarball, so its name is irrelevant.

>> In addition, several file names in the m4/ subdirectory clash after
>> 8+3 truncation:
>> gnulib-cache.m4  gnulib-common.m4  gnulib-comp.m4

> Again, the Emacs trunk already has several instances of truncation
> after 8+3 limits, such as lisp/org/org-compat.el versus
> lisp/org/org-complete.el,

Good point.  I guess Eli hadn't noticed it yet.

> and test/cedet/semantic-ia-utest.el
> versus test/cedet/semantic-tests.el, and I don't see why
> files imported from gnulib would be different.

Here, again, the `test' subdir is not included in the tarball, so it's
not an issue.


        Stefan

Re: Files from gnulib

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  bug-gnulib <bug-gnulib@gnu.org>,  emacs-devel@gnu.org
> Date: Sun, 23 Jan 2011 22:26:52 -0500
> 
> > The Emacs trunk already has seventeen other files
> > with 2 dots in their file names, with names like
> > lisp/gnus/.dir-locals.el
> 
> I don't know how MSDOS handles names that start with dot, so either this
> is a special case that's OK, or it's indeed a problem which we hadn't
> noticed yet.

The program used to unpack the .tar.gz archives automatically renames
ant .FOO files to _FOO while unpacking, so that's not a problem.
(This program, called djtar.exe, is part of the DJGPP package which is
used to build Emacs on MSDOS.)

> > and admin/charsets/mapfiles/symbol.txt.gz.
> 
> This one, OTOH is OK because the `admin' subdir is not included in the
> tarball, so its name is irrelevant.

Right.

> > Again, the Emacs trunk already has several instances of truncation
> > after 8+3 limits, such as lisp/org/org-compat.el versus
> > lisp/org/org-complete.el,
> 
> Good point.  I guess Eli hadn't noticed it yet.

I did notice that.  These files appeared very recently, and I didn't
yet have time to talk to Org mode maintainers about renaming them.
In the past a couple of other Org files were renamed for this reason,
at my request.

> > and test/cedet/semantic-ia-utest.el
> > versus test/cedet/semantic-tests.el, and I don't see why
> > files imported from gnulib would be different.
> 
> Here, again, the `test' subdir is not included in the tarball, so it's
> not an issue.

Right.

Re: Files from gnulib

[Paul Eggert]

On 01/23/11 20:01, Eli Zaretskii wrote:
> The program used to unpack the .tar.gz archives automatically renames
> ant .FOO files to _FOO while unpacking, so that's not a problem.

OK, so it sounds like there's already part of a solution here,
in that files are automatically renamed to avoid the MS-DOS
restrictions.

How about if we expand on that solution, as follows:

* Just before creating a tarball for MS-DOS, apply
  the following substitutions to the contents of all
  files that go into the tarball:

	s/c++defs\.h/cxxdefs\.h/g
	s/\([a-zA-Z0-9_]*\)\.in\.h/_\1.h/g

  The latter substitution, for example, replaces all
  instances of unistd.in.h with _unistd.h.

* Similarly, just before creating the tarball, rename
  all the source files according to the above patterns.

* The above can be done in the make-dist implementation
  for MS-DOS.

* The conflicting names in m4/* don't matter, for the same
  reason that conflicts in admin/* and tests/* don't matter:
  these files are not used in an MS-DOS build.

With this approach, the problem of MS-DOS names is handled
entirely in the MS-DOS port.

Re: Files from gnulib

[Eli Zaretskii]

> Date: Mon, 24 Jan 2011 15:26:06 -0800
> From: Paul Eggert <eggert@cs.ucla.edu>
> CC: Stefan Monnier <monnier@iro.umontreal.ca>, bug-gnulib@gnu.org, 
>  emacs-devel@gnu.org
> 
> On 01/23/11 20:01, Eli Zaretskii wrote:
> > The program used to unpack the .tar.gz archives automatically renames
> > ant .FOO files to _FOO while unpacking, so that's not a problem.
> 
> OK, so it sounds like there's already part of a solution here,
> in that files are automatically renamed to avoid the MS-DOS
> restrictions.
> 
> How about if we expand on that solution, as follows:
> 
> * Just before creating a tarball for MS-DOS, apply
>   the following substitutions to the contents of all
>   files that go into the tarball:

There are no special tarballs for MSDOS.  People use the tarball from
the GNU FTP site.  So this solution will not work.

> * The conflicting names in m4/* don't matter, for the same
>   reason that conflicts in admin/* and tests/* don't matter:
>   these files are not used in an MS-DOS build.

The files in m4/ still matter because you need to unpack the tarball,
and the utility that does that won't silently overwrite files due to
file-name clashes.

Re: Files from gnulib

[Paul Eggert]

On 01/24/2011 08:00 PM, Eli Zaretskii wrote:
> There are no special tarballs for MSDOS.

Ah, sorry, then I misunderstood.  Well, then, instead of
doing the stuff I mentioned before the tarball is created,
we can add a shell script to be run after the tarball is extracted.
For example, on MS-DOS the c++defs.h file is automatically renamed
to cxxdefs.h by the extractor, so the script can uniformly substitute
"cxxdefs.h" for "c++defs.h" in all the text files.  Hopefully
a similar idea works for all the other files with non-MS-DOS
names.

I assume that the "configure" procedure for MS-DOS is already
different, so this new script can be folded into that procedure.

> The files in m4/ still matter because you need to unpack the tarball,
> and the utility that does that won't silently overwrite files due to
> file-name clashes.

That's OK.  People can ignore those diagnostics, just as I assume
they already ignore the diagnostics for the files whose names
start with ".".  It doesn't matter whether those files are
extracted correctly, as the MS-DOS build doesn't use them.

Re: Files from gnulib

[Eli Zaretskii]

> Date: Tue, 25 Jan 2011 00:48:12 -0800
> From: Paul Eggert <eggert@cs.ucla.edu>
> CC: monnier@iro.umontreal.ca, bug-gnulib@gnu.org, emacs-devel@gnu.org
> 
> instead of doing the stuff I mentioned before the tarball is
> created, we can add a shell script to be run after the tarball is
> extracted.  For example, on MS-DOS the c++defs.h file is
> automatically renamed to cxxdefs.h by the extractor, so the script
> can uniformly substitute "cxxdefs.h" for "c++defs.h" in all the text
> files.  Hopefully a similar idea works for all the other files with
> non-MS-DOS names.

The magic coded into the extractor utility is limited.  It indeed
handles file names with `+' in them, but does not handle arbitrary
file names with multiple dots as a human would.  It uses some
convoluted algorithm to replace the extra dot with a `_' or a `-';
sometimes it replaces the first dot, sometimes the second.  The
results are often unpredictable or surprising, especially if, as it
often happens, the modified names also clash in the 8+3 namespace (see
below).

So going this way means a much more complex and error-prone
arrangement than a one-time rename of a small number of files.

> I assume that the "configure" procedure for MS-DOS is already
> different

Yes, see config.bat in the top-level directory.  But that's not the
issue here.

> > The files in m4/ still matter because you need to unpack the tarball,
> > and the utility that does that won't silently overwrite files due to
> > file-name clashes.
> 
> That's OK.  People can ignore those diagnostics

The result is not ignorable diagnostics, but a prompt for the user to
provide an alternate name.  Since the user does not generally know
whether these files are needed by the build, she will not be able to
deal with the prompt.

> just as I assume they already ignore the diagnostics for the files
> whose names start with ".".

No, there are no diagnostics for these conversions, they are done
silently.

Re: Files from gnulib

[Paul Eggert]

On 01/25/11 03:24, Eli Zaretskii wrote:

> It uses some
> convoluted algorithm to replace the extra dot with a `_' or a `-';
> sometimes it replaces the first dot, sometimes the second.  The
> results are often unpredictable or surprising, especially if, as it
> often happens, the modified names also clash in the 8+3 namespace (see
> below).

The results may seem convoluted, but I doubt whether they are actually
unpredictable.  I suspect we can accommodate whatever programmatic scheme
djtar defaults to.  Or we can tell djtar to use our own scheme, as
discussed below.

>> That's OK.  People can ignore those diagnostics
> 
> The result is not ignorable diagnostics, but a prompt for the user to
> provide an alternate name.  Since the user does not generally know
> whether these files are needed by the build, she will not be able to
> deal with the prompt.

We already provide instructions for people who want to build Emacs
on MS-DOS platforms.  Our instructions can be expanded slightly
to tell them how to extract the files in the first place, and how
to deal with such prompts.  They can be asked to just type RETURN,
for example.

Or, if that's too much trouble, the MS-DOS build instructions could
instead tell people to fetch a small file "emacs-25.chg", and then
execute

   djtar -n emacs-25.chg emacs-25.tgz

where emacs-25.chg is a list of desired file name conversions.
That is not much trouble, and it also addresses the extraction
name-change issues that have been raised so far.  If we don't
want to distribute this small file separately, we can bundle it
as part of the Emacs tarball, and give instructions on how to
extract it separately first.

> So going this way means a much more complex and error-prone
> arrangement than a one-time rename of a small number of files.

It is a bit more complex and error-prone for MS-DOS, yes.  But
it has the advantage of compartmentalizing the MS-DOS restrictions
into the MS-DOS build instructions, rather than having these restrictions
spread to Emacs more generally, where they complicate software
integration efforts there.  There is a significant advantage to
modularization and separation of concerns, one that outweighs minor
increases in complexity for individual components.

Re: Files from gnulib

[Eli Zaretskii]

> Date: Tue, 25 Jan 2011 10:07:45 -0800
> From: Paul Eggert <eggert@cs.ucla.edu>
> CC: monnier@iro.umontreal.ca, bug-gnulib@gnu.org, emacs-devel@gnu.org
> 
> We already provide instructions for people who want to build Emacs
> on MS-DOS platforms.  Our instructions can be expanded slightly
> to tell them how to extract the files in the first place, and how
> to deal with such prompts.  They can be asked to just type RETURN,
> for example.

To read the instructions, you need to unpack the archive first.

>    djtar -n emacs-25.chg emacs-25.tgz
> ...
> It is a bit more complex and error-prone for MS-DOS, yes.  But
> it has the advantage of compartmentalizing the MS-DOS restrictions
> into the MS-DOS build instructions

Sorry, but these complications are unacceptable for me.  We use
something like that in GDB, and the result is extremely fragile and
error-prone, I find problems with missing renames every time I build
GDB.  I'm not going to make the same kind or mistake again in Emacs.

Stefan and Chong, please make a decision regarding this issue.  If the
decision is not to rename these few files in the Emacs distribution,
and instead ask me to cope with these complications, I will understand
that the knee-jerk reaction of too many members of this community when
they hear "MS-DOS" is more important that any voice of reason, and I
will therefore resign from everything I do for Emacs development.

Re: Files from gnulib

[Stefan Monnier]

>> djtar -n emacs-25.chg emacs-25.tgz
>> ...
>> It is a bit more complex and error-prone for MS-DOS, yes.  But
>> it has the advantage of compartmentalizing the MS-DOS restrictions
>> into the MS-DOS build instructions

> Sorry, but these complications are unacceptable for me.  We use
> something like that in GDB, and the result is extremely fragile and
> error-prone, I find problems with missing renames every time I build
> GDB.  I'm not going to make the same kind or mistake again in Emacs.

Here's a compromise:
- let's do the renames that need to be done (they're really not
  problematic, AFAICT).
- let's try and get this "djtar -n emacs-25.chg emacs-25.tgz" to work
  reliably, e.g. by auto-building the emacs-25.chg file as part of
  make-dist, so there's no way for that file to be "missing renames".


        Stefan

Re: Files from gnulib

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: Paul Eggert <eggert@cs.ucla.edu>, cyd@stupidchicken.com,
>         bug-gnulib@gnu.org, emacs-devel@gnu.org
> Date: Tue, 25 Jan 2011 16:03:16 -0500
> 
> - let's try and get this "djtar -n emacs-25.chg emacs-25.tgz" to work
>   reliably, e.g. by auto-building the emacs-25.chg file as part of
>   make-dist, so there's no way for that file to be "missing renames".

Based on the experience from GDB, here are the problems with this
approach:

  . The remapping of file names cannot be fully automated.  Once all
    limits on file names are removed ("we don't want even the
    slightest impediment", says Jim), sooner or later you get to the
    point where the number of files to be renamed becomes large and a
    human needs to define how files will be renamed, because you want
    names that at least remotely resemble the original ones, but still
    don't clash in the 8+3 namespace.  (I hope no one is seriously
    entertaining the idea of producing meaningless names like
    foo~123.c or ABC123EF.c, from some hash or whatever).

  . Once the remapping is maintained by humans, it becomes unreliable.

  . The unpacking instructions are part of the tarball, and need to be
    extracted separately before the "main" extraction begins.  More
    importantly, the remapping file needs to be extracted before that
    as well.  This makes the entire unpacking procedure extremely
    complicated and thus error-prone, except if the person who does
    that is the one who designed it and wrote the instructions in the
    first place.

In addition, there will be a need to deal with something that the GDB
distribution doesn't.  In GDB, the files that are renamed during
unpacking are only those that are not used for the DOS build.  By
contrast, here we want to rename files that are used during the build.
So there will be a need to edit all the files that reference the
renamed ones, before the build can begin.  This can be done with Sed,
but the DOS shell doesn't have any way of recursively descending
through a directory tree.  So, if we want to make this editing as part
of config.bat, the subdirectories of the Emacs tree will have to be
hard-coded in config.bat, which is yet another source of errors, when
a subdirectory is added or removed.  (The alternative, of using a port
of GNU Find, would mean addition of another tool that is not required
today for building Emacs.)

Perhaps it's possible to solve all this in a satisfactory manner, but
doing so would require a lot of work, much more than I am willing to
invest.  (I still want to finish development of bidirectional editing,
remember?)  Given the small (yes, small!) fraction of files that ever
need to be renamed, the significant effort needed for getting this
particular alternative right is not justified.

Re: Files from gnulib

[Stefan Monnier]

> Based on the experience from GDB, here are the problems with this
> approach:

>   . The remapping of file names cannot be fully automated.  Once all
>     limits on file names are removed ("we don't want even the
>     slightest impediment", says Jim), sooner or later you get to the
>     point where the number of files to be renamed becomes large and a
>     human needs to define how files will be renamed, because you want
>     names that at least remotely resemble the original ones, but still
>     don't clash in the 8+3 namespace.  (I hope no one is seriously
>     entertaining the idea of producing meaningless names like
>     foo~123.c or ABC123EF.c, from some hash or whatever).

Yes, I was assuming that there would still be few files, and that the
renaming would be human-controlled (but the make-dist script signals an
error if the human-provided rules don't cover all cases).  So it
wouldn't help for cases like CEDET.

>   . Once the remapping is maintained by humans, it becomes unreliable.

Why is that?

>   . The unpacking instructions are part of the tarball, and need to be
>     extracted separately before the "main" extraction begins.  More
>     importantly, the remapping file needs to be extracted before that
>     as well.  This makes the entire unpacking procedure extremely
>     complicated and thus error-prone, except if the person who does
>     that is the one who designed it and wrote the instructions in the
>     first place.

Rather than distribute a file that needs to be passed to djtar, I was
thinking of distributing a script tailored to MS-DOS, run instead of
djtar, and which would run djtar insternally.  So this script can
provide the instructions.

> In addition, there will be a need to deal with something that the GDB
> distribution doesn't.  In GDB, the files that are renamed during
> unpacking are only those that are not used for the DOS build.  By
> contrast, here we want to rename files that are used during the build.

Yes, that's a much bigger problem.

In the mean time, please rename the files, thank you,


        Stefan

Re: Files from gnulib

[Paul Eggert]

On 01/25/11 14:15, Stefan Monnier wrote:
> In the mean time, please rename the files, thank you,

I'm afraid it's not as simple as merely renaming the files.
One must also change the contents of all files referring to
the renamed files, and change Makefile.in so that when
fresh copies are imported from gnulib, the files' names
and contents are consistently renamed.  And in the end,
one will end up with files that differ from gnulib,
which will make it a bit harder to port patches from Emacs
back to gnulib.

I'd like to pursue the idea of renaming the files only
on the MS-DOS platform.  This approach shouldn't take
that much work now, and it should save time in the future
(including lessening the need to discuss 8+3 issues on
this mailing list :-).

Re: Files from gnulib

[Eli Zaretskii]

> Date: Tue, 25 Jan 2011 17:05:28 -0800
> From: Paul Eggert <eggert@cs.ucla.edu>
> CC: Eli Zaretskii <eliz@gnu.org>, cyd@stupidchicken.com, 
>  bug-gnulib@gnu.org, emacs-devel@gnu.org
> 
> I'm afraid it's not as simple as merely renaming the files.
> One must also change the contents of all files referring to
> the renamed files, and change Makefile.in so that when
> fresh copies are imported from gnulib, the files' names
> and contents are consistently renamed.

These are all parts of the job that needs to be done under your
suggestion as well--to rename and edit when Emacs is built on MSDOS.

> I'd like to pursue the idea of renaming the files only
> on the MS-DOS platform.  This approach shouldn't take
> that much work now, and it should save time in the future

That approach as suggested (modeled on what GDB does) is unacceptable,
sorry.  I already said what it means for me personally if it will be
accepted here.

Re: Files from gnulib

[Eli Zaretskii]

> Date: Wed, 26 Jan 2011 06:12:55 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: cyd@stupidchicken.com, bug-gnulib@gnu.org, monnier@IRO.UMontreal.CA,
> 	emacs-devel@gnu.org
> 
> > I'd like to pursue the idea of renaming the files only
> > on the MS-DOS platform.  This approach shouldn't take
> > that much work now, and it should save time in the future
> 
> That approach as suggested (modeled on what GDB does) is unacceptable,
> sorry.

Here's a compromise that at least I can live with; hope others can,
too:

 1) For c++defs.h and the lib/*.in.h files: rely on the automatic
    renaming by djtar.  Files that reference those will be edited by
    config.bat as part of configuring Emacs for the MS-DOS build.

 2) For the 3 m4/gnulib-*.m4 files: rename them.  I suggested one way
    of renaming, but there's nothing sacred about that; any renaming
    that will get rid of the name clash is fine with me.

As long as the list of files that get handled by 1) is relatively
small and kept under control, this is manageable.  This means no "open
season" for disregarding the issue altogether, as in "we already have
such problems elsewhere, why not add a few more".

I hope that as long as the list of files that are handled by 2) is
short (3 for now), that would be regarded as manageable and acceptable
by gnulib people.

[Make no mistake: supporting 1) is still a complication.  There's more
there than meets the eye.  For example, when djtar runs on Windows, it
doesn't perform any automatic renames, because it knows that long file
names are available (DJGPP has built-in support for them in the
Standard C library).  So, to support the Emacs build on both plain DOS
and Windows, config.bat will need to do slightly different things in
each case, which means more scripting and more testing.  But these
complications are relatively minor, don't affect the end users, and I
have good experience with solving such problems, in Emacs and
elsewhere.]

Proposed gnulib renames [was: Files from gnulib]

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 3980 bytes --]

[intentionally breaking threading and retitling this, to try and make it
easier to see replies to just this aspect of the thread]

On 01/25/2011 11:01 PM, Eli Zaretskii wrote:
> Here's a compromise that at least I can live with; hope others can,
> too:
> 
>  1) For c++defs.h and the lib/*.in.h files: rely on the automatic
>     renaming by djtar.  Files that reference those will be edited by
>     config.bat as part of configuring Emacs for the MS-DOS build.

Paul already suggested renaming c++defs.h to cxxdefs.h in gnulib, which
makes sense to me in light of POSIX restrictions on portable filenames;
however, this module belongs to Bruno, so it is his call.

As for the *.in.h files, the ONLY other files that refer to those at
build time are the Makefile snippets in module files that convert them
over to *.h replacement headers.  Here's a link to some of the
back-discussion where we first settled on the name *.in.h in the first
place in Oct 2007 (we were previously using *_.h, but underscores are
painful to type in daily use; and also on the table at the time was the
rejected idea of *.h.in and *.hin):

http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/11185/focus=11206

and given that the renaming from *_.h -> *.in.h was practically
mechanical, a conversion from *.in.h -> *-in.h would likewise be
mechanical, but I'm not sure whether to make that jump yet:

http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/11182/focus=11352

that thread also included a quote from Eli, predicting the death of an
emacs DOS build (for good or for bad, that prediction has not come true):

http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/11182/focus=11234

> 
>  2) For the 3 m4/gnulib-*.m4 files: rename them.  I suggested one way
>     of renaming, but there's nothing sacred about that; any renaming
>     that will get rid of the name clash is fine with me.

This would involve a change mainly to gnulib-tool (2 of the 3 gnulib-c*
files are generated; there's also gnulib-tool.m4 to avoid clashing
with).  Changing gnulib-cache.m4 would have downstream effects on all
gnulib clients; it could be done with a NEWS entry, but I'd rather avoid
that.  But the other two files (gnulib-common.m4 and gnulib-comp.m4)
seem like fair game; how about the names gnulib-prereq.m4 (since all the
common code is prerequisite to the rest of gnulib) and gnulib-list.m4
(since comp contains the computed list of files/macros installed by
gnulib-tool).

> As long as the list of files that get handled by 1) is relatively
> small and kept under control, this is manageable.

Right now, there is the c++defs naming issue (with user-visible changes
to all gnulib clients, but worth making).

Also, there are currently 63 *.in.h files available in gnulib (I'm not
sure how many will ever be used by emacs), but I'd rather see the same
change made to all 63 files (and their corresponding modules) at once if
the change is made upstream in gnulib.  I agree that use of gnulib-tool
--local-dir won't quite work; it could easily patch the module
descriptions (and makefile snippets) to refer to *-in.h files with
minimal risk of too many upstream changes to track, but the rename
itself is not possible via simple patch(1) files (yes, GNU patchutils is
adding support for git-style rename patches, which would be much easier
to maintain, but I don't know if we should rely on that yet).  But if
gnulib makes no change to these 63 files, then that's an upper bound on
the possible complexity of Eli hand-maintaining the rename within the
DOS build of emacs.

> I hope that as long as the list of files that are handled by 2) is
> short (3 for now), that would be regarded as manageable and acceptable
> by gnulib people.

It seems reasonable to me, but I'm not the primary author of
gnulib-tool.  Bruno?

-- 
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: Proposed gnulib renames

[Eli Zaretskii]

> Date: Wed, 26 Jan 2011 08:19:13 -0700
> From: Eric Blake <eblake@redhat.com>
> CC: cyd@stupidchicken.com, eggert@cs.ucla.edu, bug-gnulib@gnu.org,
>         monnier@IRO.UMontreal.CA, emacs-devel@gnu.org,
>         Bruno Haible <bruno@clisp.org>
> 
> >  1) For c++defs.h and the lib/*.in.h files: rely on the automatic
> >     renaming by djtar.  Files that reference those will be edited by
> >     config.bat as part of configuring Emacs for the MS-DOS build.
> 
> Paul already suggested renaming c++defs.h to cxxdefs.h in gnulib, which
> makes sense to me in light of POSIX restrictions on portable filenames;
> however, this module belongs to Bruno, so it is his call.

And Bruno already voiced his objections, so I guess I will have to
live will that.  It's not a big deal to edit with Sed the few files
that refer to c++defs.h, as part of the config.bat run.  Of course, if
Bruno will change his mind, it will be even better.

> and given that the renaming from *_.h -> *.in.h was practically
> mechanical, a conversion from *.in.h -> *-in.h would likewise be
> mechanical, but I'm not sure whether to make that jump yet:
> 
> http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/11182/focus=11352

I'm okay with renaming them to use underscores.  But if gnulib
maintainers object, the djtar utility will take care of these, too.
Editing Makefile.in to refer to them in config.h should not be a big
issue.

> that thread also included a quote from Eli, predicting the death of an
> emacs DOS build (for good or for bad, that prediction has not come true):

Life has its surprises.  I found some free time to work on support for
bidirectional editing in Emacs, and as a side effect, I could invest
some of that time into reviving the DOS build.

> >  2) For the 3 m4/gnulib-*.m4 files: rename them.  I suggested one way
> >     of renaming, but there's nothing sacred about that; any renaming
> >     that will get rid of the name clash is fine with me.
> 
> This would involve a change mainly to gnulib-tool (2 of the 3 gnulib-c*
> files are generated; there's also gnulib-tool.m4 to avoid clashing
> with).  Changing gnulib-cache.m4 would have downstream effects on all
> gnulib clients; it could be done with a NEWS entry, but I'd rather avoid
> that.  But the other two files (gnulib-common.m4 and gnulib-comp.m4)
> seem like fair game; how about the names gnulib-prereq.m4 (since all the
> common code is prerequisite to the rest of gnulib) and gnulib-list.m4
> (since comp contains the computed list of files/macros installed by
> gnulib-tool).

Of course, it is enough to rename only 2 of the 3, to avoid file-name
clashes.  So if your suggestion to rename gnulib-common.m4 and
gnulib-comp.m4 is accepted, that would solve the problem entirely.

Thanks!

Re: Proposed gnulib renames

[Bruno Haible]

Hi Eric, Eli,

Remember that Eli said that the entire discussion is about renaming files
in the Emacs repository, not in gnulib [1]. I also pointed out that MSDOS
support is outside of gnulib's scope [2].

On this background, for me the goal is not to do any renamings in gnulib,
but rather to help the Emacs maintainers do these renamings easily when
they pull from gnulib.


1) About c++defs.h:

I have proposed in [2] a modification to the Emacs sources that will ensure
that
  - No c++defs.h is contained in the build.
  - A gnulib-local/modules/c++defs.orig file is contained in the tarball but
    not used in the build. According to Eli [3][4] the djtar program can
    deal with such a file automatically.


2) About *.in.h include files:

Eric Blake wrote:
> I agree that use of gnulib-tool
> --local-dir won't quite work; it could easily patch the module
> descriptions (and makefile snippets) to refer to *-in.h files with
> minimal risk of too many upstream changes to track, but the rename
> itself is not possible via simple patch(1) files (yes, GNU patchutils is
> adding support for git-style rename patches, which would be much easier
> to maintain, but I don't know if we should rely on that yet).

The simplest way to deal with this is that the script for updating the
Emacs sources from gnulib
  1. runs "gnulib-tool"
  2. looks at list of lib/*.in.h files,
  3. performs some 'mv' commands to rename them,
  4. performs some sed replacements on the generated Makefile.am.
Compared to the sed processing that is done by gnulib-tool, this is really
minor.

Eli Zaretskii says in [5]:

> I'm okay with renaming them to use underscores.  But if gnulib
> maintainers object, the djtar utility will take care of these, too.
> Editing Makefile.in to refer to them in config.h should not be a big
> issue.

I agree with Eli that a bit of hand-written renaming in the script
is feasible and maintainable.


3) About the collision of gnulib-cache.m4, gnulib-comp.m4, gnulib-common.m4:

Only one of these files is in gnulib. The other two are generated, and these
files are used by two programs: 'gnulib-tool' and 'aclocal'. For
'gnulib-tool', the file names matter, but 'gnulib-tool' is only invoked
from a specific script or Makefile in the Emacs sources. For 'aclocal',
the names of the .m4 files don't matter.

So, I would propose that you change the script from

   gnulib-tool ...<many options>

to

   if test -f m4/gl-cache.m4; then mv m4/gl-cache.m4 m4/gnulib-cache.m4; done
   if test -f m4/gl-comp.m4; then mv m4/gl-comp.m4 m4/gnulib-comp.m4; done
   gnulib-tool ...<many options>
   mv m4/gnulib-cache.m4 m4/gl-cache.m4
   mv m4/gnulib-comp.m4 m4/gl-comp.m4

Eric Blake wrote:
> Changing gnulib-cache.m4 would have downstream effects on all
> gnulib clients; it could be done with a NEWS entry, but I'd rather avoid
> that.  But the other two files (gnulib-common.m4 and gnulib-comp.m4)
> seem like fair game; how about the names gnulib-prereq.m4 (since all the
> common code is prerequisite to the rest of gnulib) and gnulib-list.m4
> (since comp contains the computed list of files/macros installed by
> gnulib-tool).

It is out of question to rename things in a way that would affect all
gnulib users. If necessary, we could introduce a special option or two
in gnulib-tool, for Emacs developers to use. But even this is not needed,
given that it can be done with two 'mv' commands before and two 'mv'
commands after the invocation of gnulib-tool.


Eli, do these three proposals solve the issues? If not, what's remaining?

Bruno


[1] http://lists.gnu.org/archive/html/emacs-devel/2011-01/msg00865.html
[2] http://lists.gnu.org/archive/html/emacs-devel/2011-01/msg00779.html
[3] http://lists.gnu.org/archive/html/emacs-devel/2011-01/msg00827.html
[4] http://lists.gnu.org/archive/html/emacs-devel/2011-01/msg00866.html
[5] http://lists.gnu.org/archive/html/emacs-devel/2011-01/msg00922.html

Re: Proposed gnulib renames

[Eli Zaretskii]

> From: Bruno Haible <bruno@clisp.org>
> Date: Wed, 26 Jan 2011 18:33:20 +0100
> Cc: cyd@stupidchicken.com,
>  eggert@cs.ucla.edu,
>  bug-gnulib@gnu.org,
>  monnier@iro.umontreal.ca,
>  emacs-devel@gnu.org
> 
> Remember that Eli said that the entire discussion is about renaming files
> in the Emacs repository, not in gnulib [1].

That's true.  I have no intention to force any undesired changes on
gnulib.

> 1) About c++defs.h:
> 
> I have proposed in [2] a modification to the Emacs sources that will ensure
> that
>   - No c++defs.h is contained in the build.
>   - A gnulib-local/modules/c++defs.orig file is contained in the tarball but
>     not used in the build. According to Eli [3][4] the djtar program can
>     deal with such a file automatically.

IIUC, the file gnulib-local/modules/c++defs.diff is a patch to be
applied by gnulib-tool when it reads c++defs from the gnulib
directory.  But what is the file gnulib-local/build-aux/cxxdefs.h for?

Also, the patch in gnulib-local/modules/c++defs.diff would need to be
updated from time to time, when c++defs in gnulib changes
significantly, is that right?

Finally, what is the file c++defs that will be patched by that patch?
I see no such file in Emacs at the moment.

> 2) About *.in.h include files:
> [...]
> The simplest way to deal with this is that the script for updating the
> Emacs sources from gnulib
>   1. runs "gnulib-tool"
>   2. looks at list of lib/*.in.h files,
>   3. performs some 'mv' commands to rename them,
>   4. performs some sed replacements on the generated Makefile.am.

That's fine with me, but these Sed replacements could be done by
config.bat only for the DOS build.  Of course, it's slightly more
complicated to do that with DOS shells, but it's certainly doable.

> 3) About the collision of gnulib-cache.m4, gnulib-comp.m4, gnulib-common.m4:
> [...]
> So, I would propose that you change the script from
> 
>    gnulib-tool ...<many options>
> 
> to
> 
>    if test -f m4/gl-cache.m4; then mv m4/gl-cache.m4 m4/gnulib-cache.m4; done
>    if test -f m4/gl-comp.m4; then mv m4/gl-comp.m4 m4/gnulib-comp.m4; done
>    gnulib-tool ...<many options>
>    mv m4/gnulib-cache.m4 m4/gl-cache.m4
>    mv m4/gnulib-comp.m4 m4/gl-comp.m4

That's fine with me.

> Eli, do these three proposals solve the issues?

I think so.

> If not, what's remaining?

Assuming this is acceptable to Stefan and Paul, all that's left is for
me to understand the details about which I ask above, and then do all
these changes.

Thanks a lot for taking time to describe these solutions.

Re: Proposed gnulib renames

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 2584 bytes --]

On 01/26/2011 11:40 AM, Eli Zaretskii wrote:
>> I have proposed in [2] a modification to the Emacs sources that will ensure
>> that
>>   - No c++defs.h is contained in the build.
>>   - A gnulib-local/modules/c++defs.orig file is contained in the tarball but
>>     not used in the build. According to Eli [3][4] the djtar program can
>>     deal with such a file automatically.
> 
> IIUC, the file gnulib-local/modules/c++defs.diff is a patch to be
> applied by gnulib-tool when it reads c++defs from the gnulib
> directory.  But what is the file gnulib-local/build-aux/cxxdefs.h for?

The counterpart of the c++defs.diff patch that provides the contents of
the file under its desired new name.  That is, the combination of the
two patches together performs the rename of c++defs over to cxxdefs when
the module is imported into the emacs tree.

> 
> Also, the patch in gnulib-local/modules/c++defs.diff would need to be
> updated from time to time, when c++defs in gnulib changes
> significantly, is that right?

Correct, although given the stability of c++defs, that is not likely to
be very often.

> 
> Finally, what is the file c++defs that will be patched by that patch?
> I see no such file in Emacs at the moment.

It's the file gnulib/modules/c++defs that tells gnulib-module how to
import all files related to the c++defs module.  But you are changing it
to be the cxxdefs module.  Gnulib-tool uses the combination of its own
module files, as patched by your local files, to decide what to apply to
your local tree.  After gnulib-tool is complete, you no longer need any
of the gnulib/modules files stored in the emacs tree, but just the
output.  Which means that after a all is said and done, the only
remaining file with + in the name is your instructions to gnulib-tool
(not part of the normal emacs build, but necessary for whoever reruns
the gnulib synchronization script), and everything else in the emacs
tree (which is actually part of the build) has already been renamed to
cxxdefs during the gnulib-tool synchronization.

>> If not, what's remaining?
> 
> Assuming this is acceptable to Stefan and Paul, all that's left is for
> me to understand the details about which I ask above, and then do all
> these changes.

And to make sure that in the future, none of the other 8.3 conflicts
that were pointed out by Jim end up being imported into emacs during a
subsequent gnulib-tool synchronization run.

-- 
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: Proposed gnulib renames

[Bruno Haible]

Hi Eli,

> > 1) About c++defs.h:
> > 
> > I have proposed in [2] a modification to the Emacs sources that will ensure
> > that
> >   - No c++defs.h is contained in the build.
> >   - A gnulib-local/modules/c++defs.diff file is contained in the tarball but
> >     not used in the build. According to Eli [3][4] the djtar program can
> >     deal with such a file automatically.
> 
> IIUC, the file gnulib-local/modules/c++defs.diff is a patch to be
> applied by gnulib-tool when it reads c++defs from the gnulib
> directory.  But what is the file gnulib-local/build-aux/cxxdefs.h for?
> 
> Also, the patch in gnulib-local/modules/c++defs.diff would need to be
> updated from time to time, when c++defs in gnulib changes
> significantly, is that right?
> 
> Finally, what is the file c++defs that will be patched by that patch?
> I see no such file in Emacs at the moment.

This was an approach that meant to use gnulib-tool built-in functionality,
with no post-processing of the generated Makefile.am. The cxxdefs.h is
the replacement for c++defs.h (which would be a file with an undesired
name, needed during the build). This file and gnulib-local/modules/c++defs.diff
would indeed have to change when major changes are being done to this module,
but this is not going to happen frequently. The c++defs.diff file is a patch,
that would be stored in the Emacs repository, for a file c++defs that is
stored in the gnulib repository.

Another approach, that may be simpler to put in place, is to change the
script that invokes gnulib-tool so that it
  1) renames build-aux/c++defs.h to build-aux/cxxdefs.h
  2) post-processes the generated Makefile.am, replacing "c++defs" with
     "cxxdefs".
This approach requires post-processing, but will require less frequent
updates after changes in gnulib.

> > 2) About *.in.h include files:
> > [...]
> > The simplest way to deal with this is that the script for updating the
> > Emacs sources from gnulib
> >   1. runs "gnulib-tool"
> >   2. looks at list of lib/*.in.h files,
> >   3. performs some 'mv' commands to rename them,
> >   4. performs some sed replacements on the generated Makefile.am.
> 
> That's fine with me, but these Sed replacements could be done by
> config.bat only for the DOS build.  Of course, it's slightly more
> complicated to do that with DOS shells, but it's certainly doable.

I was under the impression that you wanted to have this renaming done
already in the tarball and, in consequence, also in the Emacs repository.
If the renaming can be limited to the DOS based checkouts, that's of course
more work for you but better for everyone else.

Bruno

Re: Proposed gnulib renames

[Eli Zaretskii]

> From: Bruno Haible <bruno@clisp.org>
> Date: Wed, 26 Jan 2011 20:01:30 +0100
> Cc: eblake@redhat.com,
>  cyd@stupidchicken.com,
>  eggert@cs.ucla.edu,
>  bug-gnulib@gnu.org,
>  monnier@iro.umontreal.ca,
>  emacs-devel@gnu.org
> 
> > >   1. runs "gnulib-tool"
> > >   2. looks at list of lib/*.in.h files,
> > >   3. performs some 'mv' commands to rename them,
> > >   4. performs some sed replacements on the generated Makefile.am.
> > 
> > That's fine with me, but these Sed replacements could be done by
> > config.bat only for the DOS build.  Of course, it's slightly more
> > complicated to do that with DOS shells, but it's certainly doable.
> 
> I was under the impression that you wanted to have this renaming done
> already in the tarball and, in consequence, also in the Emacs repository.

Files whose names are FOOBAR.in.h are automatically renamed by djtar,
the unpacking utility, to something like FOOBAR.in-h (and will appear
as foobar.in- on plain DOS), so they are no problem.  What I wanted to
avoid were files whose name clash after 8+3 truncation, because these
are not renamed automatically, but need user intervention for
providing an alternative name, something end users will be stumped
about.

Re: Proposed gnulib renames

[Paul Eggert]

On 01/26/2011 09:33 AM, Bruno Haible wrote:

> I agree with Eli that a bit of hand-written renaming in the script
> is feasible and maintainable.

A bit is, yes.  However, we should strive to put the burden of
renaming-for-DOS into the DOS-related source files, and so this bit
shouldn't be precedent for much more renaming.  On the contrary, I'd
like to remove this bit once a more-general solution can be found.

>    if test -f m4/gl-cache.m4; then mv m4/gl-cache.m4 m4/gnulib-cache.m4; done
>    if test -f m4/gl-comp.m4; then mv m4/gl-comp.m4 m4/gnulib-comp.m4; done
>    gnulib-tool ...<many options>
>    mv m4/gnulib-cache.m4 m4/gl-cache.m4
>    mv m4/gnulib-comp.m4 m4/gl-comp.m4

Thanks for the suggestion.  We can simplify that part further as shown
in the patch below (which I just put into the Emacs trunk).  There's
no need to keep gnulib-cache.m4, since gnulib-tool is invoked only
from Makefile.in.

However, some extra stuff is needed as well, as we need to modify
ACLOCAL_INPUTS to reflect the MS-DOS-oriented name for the renamed
file.

=== modified file 'ChangeLog'
--- ChangeLog	2011-01-26 08:36:39 +0000
+++ ChangeLog	2011-01-27 07:26:48 +0000
@@ -1,3 +1,18 @@
+2011-01-27  Paul Eggert  <eggert@cs.ucla.edu>
+
+	fix two m4/gnulib-*.m4 file names that clashed under MS-DOS
+	* Makefile.in (DOS-gnulib-comp.m4): New macro.
+	(sync-from-gnulib): Rename m4/gnulib-comp.m4 to m4/gl-comp.m4 to avoid
+	problems with MS-DOS 8+3 file name restrictions.
+	Remove m4/gnulib-cache.m4, as we can live without it.  If we kept
+	it, it would also cause problems when extracting Emacs distribution
+	tarballs on MS-DOS hosts.
+	(ACLOCAL_INPUTS): Adjust to file renaming.
+	* aclocal.m4, configure, lib/Makefile.in, src/config.in: Regenerate.
+	* config.guess, config.sub: Sync from gnulib.
+	* m4/gnulib-cache.m4: Remove from repository.
+	* m4/gl-comp.m4: Rename from m4/gnulib-comp.m4.
+
 2011-01-25  Glenn Morris  <rgm@gnu.org>
 
 	* README: Add a note about ranges in copyright years.

=== modified file 'Makefile.in'
--- Makefile.in	2011-01-26 08:36:39 +0000
+++ Makefile.in	2011-01-27 07:26:48 +0000
@@ -324,6 +324,9 @@
 $(gnulib_srcdir):
 	git clone git://git.savannah.gnu.org/gnulib.git $@
 
+# A shorter name that satisfies MS-DOS 8+3 constraints.
+DOS-gnulib-comp.m4 = gl-comp.m4
+
 # Update modules from gnulib, for maintainers, who should have it in
 # $(gnulib_srcdir) (relative to $(srcdir) and should have build tools
 # as per $(gnulib_srcdir)/DEPENDENCIES.
@@ -333,7 +336,8 @@
 sync-from-gnulib: $(gnulib_srcdir)
 	cd $(srcdir) && \
 	  $(gnulib_srcdir)/gnulib-tool $(GNULIB_TOOL_FLAGS) $(GNULIB_MODULES)
-	rm $(srcdir)/m4/warn-on-use.m4
+	cd $(srcdir)/m4 && rm gnulib-cache.m4 warn-on-use.m4
+	cd $(srcdir)/m4 && mv gnulib-comp.m4 $(DOS-gnulib-comp.m4)
 	cp $(gnulib_srcdir)/build-aux/texinfo.tex $(srcdir)/doc/misc
 	cp \
 	  $(gnulib_srcdir)/build-aux/config.sub \
@@ -406,7 +410,7 @@
 $(srcdir)/configure: $(AUTOCONF_INPUTS)
 	cd ${srcdir} && autoconf
 
-ACLOCAL_INPUTS = @MAINT@ $(srcdir)/m4/gnulib-comp.m4
+ACLOCAL_INPUTS = @MAINT@ $(srcdir)/m4/$(DOS-gnulib-comp.m4)
 $(srcdir)/aclocal.m4: $(ACLOCAL_INPUTS)
 	cd $(srcdir) && aclocal -I m4
 

[The remaining part of the patch is automatically generated.]

Re: Proposed gnulib renames

[Eli Zaretskii]

> Date: Wed, 26 Jan 2011 23:37:29 -0800
> From: Paul Eggert <eggert@cs.ucla.edu>
> CC: Eli Zaretskii <eliz@gnu.org>, Eric Blake <eblake@redhat.com>, 
>  cyd@stupidchicken.com, bug-gnulib@gnu.org, monnier@iro.umontreal.ca, 
>  emacs-devel@gnu.org
> 
> @@ -333,7 +336,8 @@
>  sync-from-gnulib: $(gnulib_srcdir)
>  	cd $(srcdir) && \
>  	  $(gnulib_srcdir)/gnulib-tool $(GNULIB_TOOL_FLAGS) $(GNULIB_MODULES)
> -	rm $(srcdir)/m4/warn-on-use.m4
> +	cd $(srcdir)/m4 && rm gnulib-cache.m4 warn-on-use.m4
> +	cd $(srcdir)/m4 && mv gnulib-comp.m4 $(DOS-gnulib-comp.m4)
>  	cp $(gnulib_srcdir)/build-aux/texinfo.tex $(srcdir)/doc/misc
>  	cp \
>  	  $(gnulib_srcdir)/build-aux/config.sub \

Don't we need the opposite "mv" before running gnulib-tool?  Apologies
if this is a dumb question, I know almost nothing about gnulib.

Re: Proposed gnulib renames

[Eli Zaretskii]

> Date: Wed, 26 Jan 2011 23:37:29 -0800
> From: Paul Eggert <eggert@cs.ucla.edu>
> CC: Eli Zaretskii <eliz@gnu.org>, Eric Blake <eblake@redhat.com>, 
>  cyd@stupidchicken.com, bug-gnulib@gnu.org, monnier@iro.umontreal.ca, 
>  emacs-devel@gnu.org
> 
> Thanks for the suggestion.  We can simplify that part further as shown
> in the patch below (which I just put into the Emacs trunk).

Thank you.

Re: Proposed gnulib renames

[Bruno Haible]

Hi Paul,

Paul Eggert wrote:
> +       * Makefile.in (DOS-gnulib-comp.m4): New macro.

A Makefile macro with a minus sign in its name?! This is not portable
according to POSIX [1]:
  "Applications shall select macro names from the set of characters
   consisting solely of periods, underscores, digits, and alphabetics
   from the portable character set"
Likewise text is also in the autoconf documentation [2].

Bruno

[1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html
[2] http://www.gnu.org/software/autoconf/manual/html_node/Special-Chars-in-Names.html

Re: Proposed gnulib renames

[Paul Eggert]

On 01/27/2011 01:59 AM, Bruno Haible wrote:

>> +       * Makefile.in (DOS-gnulib-comp.m4): New macro.
> 
> A Makefile macro with a minus sign in its name?! This is not portable
> according to POSIX [1]:

Thanks for mentioning that.  Other Emacs makefiles had it, so I naively
assumed it was OK; but you're right, we shouldn't be using unportable
Makefile identifiers.  After fixing the above problem in my previous patch
I fixed the previously-existing identifiers, as follows.

This will most likely affect the MS-DOS build.

=== modified file 'etc/ChangeLog'
--- etc/ChangeLog	2011-01-26 08:36:39 +0000
+++ etc/ChangeLog	2011-01-28 01:52:26 +0000
@@ -1,3 +1,15 @@
+2011-01-28  Paul Eggert  <eggert@cs.ucla.edu>
+
+	Redo spelling of Makefile variables to conform to POSIX.
+	POSIX does not allow "-" in Makefile variable names.
+	Reported by Bruno Haible in
+	<http://lists.gnu.org/archive/html/emacs-devel/2011-01/msg00990.html>.
+	* refcards/Makefile (DIRED_REFCARDS_PDF): Renamed from
+	DIRED-REFCARDS_PDF.
+	(MISC_REFCARDS_PDF): Renamed from MISC-REFCARDS_PDF.
+	(SURVIVAL_CARDS_PDF): Renamed from SURVIVAL-CARDS_PDF.
+	(VIPER_CARDS_PDF): Renamed from VIPER-CARDS_PDF.
+
 2011-01-18  Glenn Morris  <rgm@gnu.org>
 
 	* PROBLEMS: Add note about svn+ssh.  (Bug#7791)

=== modified file 'etc/refcards/Makefile'
--- etc/refcards/Makefile	2011-01-25 04:08:28 +0000
+++ etc/refcards/Makefile	2011-01-28 01:52:26 +0000
@@ -28,24 +28,24 @@
 refcards_pdf: ${REFCARDS_PDF}
 refcards_ps: ${REFCARDS_PDF:.pdf=.ps}
 
-DIRED-REFCARDS_PDF = dired-ref.pdf cs-dired-ref.pdf fr-dired-ref.pdf \
+DIRED_REFCARDS_PDF = dired-ref.pdf cs-dired-ref.pdf fr-dired-ref.pdf \
 		sk-dired-ref.pdf
-dired-refcards_pdf: ${DIRED-REFCARDS_PDF}
-dired-refcards_ps: ${DIRED-REFCARDS_PDF:.pdf=.ps}
+dired-refcards_pdf: ${DIRED_REFCARDS_PDF}
+dired-refcards_ps: ${DIRED_REFCARDS_PDF:.pdf=.ps}
 
-MISC-REFCARDS_PDF = calccard.pdf gnus-booklet.pdf gnus-refcard.pdf orgcard.pdf
-misc-refcards_pdf: ${MISC-REFCARDS_PDF}
-misc-refcards_ps: ${MISC-REFCARDS_PDF:.pdf=.ps}
+MISC_REFCARDS_PDF = calccard.pdf gnus-booklet.pdf gnus-refcard.pdf orgcard.pdf
+misc-refcards_pdf: ${MISC_REFCARDS_PDF}
+misc-refcards_ps: ${MISC_REFCARDS_PDF:.pdf=.ps}
 
 
 ## The following files are not included with Emacs.
-SURVIVAL-CARDS_PDF = survival.pdf cs-survival.pdf sk-survival.pdf
-survival-cards_pdf: ${SURVIVAL-CARDS_PDF}
-survival-cards_ps: ${SURVIVAL-CARDS_PDF:.pdf=.ps}
+SURVIVAL_CARDS_PDF = survival.pdf cs-survival.pdf sk-survival.pdf
+survival-cards_pdf: ${SURVIVAL_CARDS_PDF}
+survival-cards_ps: ${SURVIVAL_CARDS_PDF:.pdf=.ps}
 
-VIPER-CARDS_PDF = vipcard.pdf viperCard.pdf
-viper-cards_pdf: ${VIPER-CARDS_PDF}
-viper-cards_ps: ${VIPER-CARDS_PDF:.pdf=.ps}
+VIPER_CARDS_PDF = vipcard.pdf viperCard.pdf
+viper-cards_pdf: ${VIPER_CARDS_PDF}
+viper-cards_ps: ${VIPER_CARDS_PDF:.pdf=.ps}
 
 
 ## PDF files.
@@ -74,7 +74,7 @@
 #gnus-logo.pdf: %.pdf: %.eps
 #	ps2pdf $<
 
-gnus-refcard.pdf: %.pdf: %.tex gnus-logo.pdf 
+gnus-refcard.pdf: %.pdf: %.tex gnus-logo.pdf
 	pdflatex $<
 
 gnus-booklet.pdf: gnus-refcard.tex gnus-logo.pdf

=== modified file 'leim/ChangeLog'
--- leim/ChangeLog	2011-01-25 04:08:28 +0000
+++ leim/ChangeLog	2011-01-28 01:52:26 +0000
@@ -1,3 +1,22 @@
+2011-01-28  Paul Eggert  <eggert@cs.ucla.edu>
+
+	Redo spelling of Makefile variables to conform to POSIX.
+	POSIX does not allow "-" in Makefile variable names.
+	Reported by Bruno Haible in
+	<http://lists.gnu.org/archive/html/emacs-devel/2011-01/msg00990.html>.
+	* Makefile.in (BUILT_EMACS): Renamed from BUILT-EMACS.
+	(TIT_GB): Renamed from TIT-GB.
+	(CHINESE_TIT): Renamed from CHINESE-TIT.
+	(NON_TIT_GB): Renamed from NON-TIT-GB.
+	(NON_TIT_BIG5): Renamed from NON-TIT-BIG5.
+	(CHINESE_NON_TIT): Renamed from CHINESE-NON-TIT.
+	(CHINESE_GB): Renamed from CHINESE-GB.
+	(CHINESE_BIG5): Renamed from CHINESE-BIG5.
+	(TIT_MISC): Renamed from TIT-MISC.
+	(NON_TIT_MISC): Renamed from NON-TIT-MISC.
+	(TIT_SOURCES): Renamed from TIT-SOURCES.
+	(MISC_SOURCES): Renamed from MISC-SOURCES.
+
 2011-01-08  Glenn Morris  <rgm@gnu.org>
 
 	* makefile.w32-in (RUN_EMACS):

=== modified file 'leim/Makefile.in'
--- leim/Makefile.in	2011-01-26 08:36:39 +0000
+++ leim/Makefile.in	2011-01-28 01:52:26 +0000
@@ -41,20 +41,20 @@
 
 # Which Emacs to use to convert TIT files to Emacs Lisp files,
 # byte-compile Emacs Lisp files, and generate the file leim-list.el.
-BUILT-EMACS = ../src/emacs
+BUILT_EMACS = ../src/emacs
 
 buildlisppath=${srcdir}/../lisp
 
 # How to run Emacs.
-RUN-EMACS = EMACSLOADPATH=$(buildlisppath) LC_ALL=C \
-	${BUILT-EMACS} -batch --no-site-file --no-site-lisp
+RUN_EMACS = EMACSLOADPATH=$(buildlisppath) LC_ALL=C \
+	${BUILT_EMACS} -batch --no-site-file --no-site-lisp
 
 # Subdirectories to be made if ${srcdir} is different from the current
 # directory.
 SUBDIRS=quail
 
 # Files generated from TIT dictionaries for Chinese GB character set.
-TIT-GB=\
+TIT_GB=\
 	quail/CCDOSPY.elc	\
 	quail/Punct.elc		\
 	quail/QJ.elc		\
@@ -72,17 +72,17 @@
 	quail/QJ-b5.elc		\
 	quail/ZOZY.elc
 
-CHINESE-TIT=${TIT-GB} ${TIT-BIG5}
-
-NON-TIT-GB=${srcdir}/quail/py-punct.elc
-
-NON-TIT-BIG5=${srcdir}/quail/pypunct-b5.elc
-
-CHINESE-NON-TIT=${NON-TIT-GB} ${NON-TIT-BIG5}
-
-CHINESE-GB=${TIT-GB} ${NON-TIT-GB}
-
-CHINESE-BIG5=${TIT-BIG5} ${NON-TIT-BIG5}
+CHINESE_TIT=${TIT_GB} ${TIT_BIG5}
+
+NON_TIT_GB=${srcdir}/quail/py-punct.elc
+
+NON_TIT_BIG5=${srcdir}/quail/pypunct-b5.elc
+
+CHINESE_NON_TIT=${NON_TIT_GB} ${NON_TIT_BIG5}
+
+CHINESE_GB=${TIT_GB} ${NON_TIT_GB}
+
+CHINESE_BIG5=${TIT_BIG5} ${NON_TIT_BIG5}
 
 JAPANESE=${srcdir}/quail/japanese.elc ${srcdir}/ja-dic/ja-dic.elc
 
@@ -138,22 +138,22 @@
 	quail/CTLau.elc		\
 	quail/CTLau-b5.elc
 
-CHINESE=${CHINESE-GB} ${CHINESE-BIG5}
+CHINESE=${CHINESE_GB} ${CHINESE_BIG5}
 EASTASIA=${CHINESE} ${JAPANESE} ${KOREAN}
 ASIA=${EASTASIA} ${THAI} ${VIETNAMESE} ${LAO} ${INDIAN} ${TIBETAN}
 EUROPEAN=${LATIN} ${SLAVIC} ${GREEK} ${RUSSIAN}
 WORLD=${ASIA} ${EUROPEAN} ${OTHERS} ${MISC} ${UNICODE}
 
-TIT-MISC=${CHINESE-TIT} ${MISC}
-NON-TIT-MISC=${CHINESE-NON-TIT} ${JAPANESE} ${KOREAN} ${EUROPEAN} ${OTHERS}
+TIT_MISC=${CHINESE_TIT} ${MISC}
+NON_TIT_MISC=${CHINESE_NON_TIT} ${JAPANESE} ${KOREAN} ${EUROPEAN} ${OTHERS}
 
 .SUFFIXES: .elc .el
 
 .el.elc:
 	@echo Compiling $<
-	@${RUN-EMACS} -f batch-byte-compile $<
+	@${RUN_EMACS} -f batch-byte-compile $<
 
-all: ${BUILT-EMACS} ${SUBDIRS} leim-list.el ${WORLD}
+all: ${BUILT_EMACS} ${SUBDIRS} leim-list.el ${WORLD}
 
 # To ensure that we can run Emacs.  This target is ignored (never
 # being hit) if a user changes default value of EMACS.
@@ -164,7 +164,7 @@
 	mkdir $@
 	touch stamp-subdir
 
-TIT-SOURCES= \
+TIT_SOURCES= \
 	${srcdir}/CXTERM-DIC/4Corner.tit \
 	${srcdir}/CXTERM-DIC/ARRAY30.tit \
 	${srcdir}/CXTERM-DIC/CCDOSPY.tit \
@@ -179,15 +179,15 @@
 	${srcdir}/CXTERM-DIC/TONEPY.tit \
 	${srcdir}/CXTERM-DIC/ZOZY.tit
 
-${CHINESE-TIT:.elc=.el}: changed.tit
+${CHINESE_TIT:.elc=.el}: changed.tit
 	@true
 
-changed.tit: ${TIT-SOURCES}
-	${RUN-EMACS} -l ${buildlisppath}/international/titdic-cnv \
+changed.tit: ${TIT_SOURCES}
+	${RUN_EMACS} -l ${buildlisppath}/international/titdic-cnv \
 	  -f batch-titdic-convert -dir quail ${srcdir}/CXTERM-DIC; \
 	  echo "changed" > $@
 
-MISC-SOURCES= \
+MISC_SOURCES= \
 	${srcdir}/MISC-DIC/CTLau-b5.html \
 	${srcdir}/MISC-DIC/CTLau.html \
 	${srcdir}/MISC-DIC/cangjie-table.b5 \
@@ -198,20 +198,20 @@
 ${MISC:.elc=.el}: changed.misc
 	@true
 
-changed.misc: ${MISC-SOURCES}
-	${RUN-EMACS} -l ${buildlisppath}/international/titdic-cnv \
+changed.misc: ${MISC_SOURCES}
+	${RUN_EMACS} -l ${buildlisppath}/international/titdic-cnv \
 	  -f batch-miscdic-convert -dir quail ${srcdir}/MISC-DIC; \
 	  echo "changed" > $@
 
-leim-list.el: ${SUBDIRS} ${TIT-MISC} changed.tit changed.misc ${srcdir}/leim-ext.el
+leim-list.el: ${SUBDIRS} ${TIT_MISC} changed.tit changed.misc ${srcdir}/leim-ext.el
 	rm -f leim-list.el
-	${RUN-EMACS}  -l ${buildlisppath}/international/quail \
-	  -f batch-byte-compile-if-not-done ${TIT-MISC:.elc=.el}
+	${RUN_EMACS}  -l ${buildlisppath}/international/quail \
+	  -f batch-byte-compile-if-not-done ${TIT_MISC:.elc=.el}
 	if [ x`(cd ${srcdir} && /bin/pwd)` = x`(/bin/pwd)` ] ; then \
-	  ${RUN-EMACS} -l ${buildlisppath}/international/quail \
+	  ${RUN_EMACS} -l ${buildlisppath}/international/quail \
 	    --eval "(update-leim-list-file \".\")" ; \
 	else \
-	  ${RUN-EMACS} -l ${buildlisppath}/international/quail \
+	  ${RUN_EMACS} -l ${buildlisppath}/international/quail \
 	    --eval "(update-leim-list-file \".\" \"${srcdir}\")" ; \
 	fi
 	sed -n '/^[^;]/ p' < ${srcdir}/leim-ext.el >> $@
@@ -264,7 +264,7 @@
 	else true ; fi
 
 clean mostlyclean:
-	rm -f ${TIT-MISC} ${TIT-MISC:.elc=.el} \
+	rm -f ${TIT_MISC} ${TIT_MISC:.elc=.el} \
 		leim-list.el changed.tit changed.misc
 
 # The following target is needed because the `clean' target only removes
@@ -287,5 +287,5 @@
 .PHONY: check-declare
 
 check-declare:
-	$(RUN-EMACS) -l $(buildlisppath)/emacs-lisp/check-declare \
+	$(RUN_EMACS) -l $(buildlisppath)/emacs-lisp/check-declare \
 	  --eval '(check-declare-directory "$(srcdir)")'

Re: Proposed gnulib renames

[Bruno Haible]

Paul Eggert wrote:
> There's no need to keep gnulib-cache.m4, since gnulib-tool is invoked only
> from Makefile.in.

I disagree. The gnulib documentation ([1], paragraph 1) recommends to put also
gnulib-cache.m4 into the repository.

The reason is that when a file is renamed in gnulib or some module dependency
is removed in gnulib, what happens at the next invocation of sync-from-gnulib?
  - When gnulib-cache.m4 is present, gnulib-tool will add a file under the
    new name and remove the old file. Because it knows that the file came from
    gnulib.
  - When gnulib-cache.m4 is missing, gnulib-tool will just add a file under
    the new name, but leave the old file around, because it looks like that
    file was genuine Emacs source.

So, if you routinely remove gnulib-cache.m4, over time the repository will
accumulate garbage files. Sometimes they don't hurt (if it's just a .c file),
but it can really get in the way and cause trouble (if it's a .h file).

Bruno

[1] http://www.gnu.org/software/gnulib/manual/html_node/VCS-Issues.html

Re: Proposed gnulib renames

[Bruno Haible]

> I disagree. The gnulib documentation ([1], paragraph 1) recommends to put also
> gnulib-cache.m4 into the repository.
> 
> The reason is that when a file is renamed in gnulib or some module dependency
> is removed in gnulib, what happens at the next invocation of sync-from-gnulib?

Oops, I misspoke. The gnulib documentation does recommend to put
gnulib-cache.m4 into the repository, but this is only a convenience for
someone who wants to use the --add-import or --remove-import options
and therefore not really necessary. The file that needs to be kept so
that old files don't accumulate as garbage is gnulib-comp.m4.

Sorry for the confusion.

Bruno

Re: Proposed gnulib renames

[Paul Eggert]

On 01/27/11 02:23, Bruno Haible wrote:
> The file that needs to be kept so
> that old files don't accumulate as garbage is gnulib-comp.m4.

Thanks, I installed this into the Emacs trunk.  I was leery about
using 'mv' as the first step, because what if 'make' is interrupted
right after the 'mv'? gl-comp.m4 will be missing and a later
'make' will fail because aclocal.m4 can't be built.  So I instead
used 'cp', as follows.

=== modified file 'ChangeLog'
--- ChangeLog	2011-01-27 07:24:57 +0000
+++ ChangeLog	2011-01-28 00:25:24 +0000
@@ -1,3 +1,12 @@
+2011-01-28  Paul Eggert  <eggert@cs.ucla.edu>
+
+	improve fix for MS-DOS file name clash
+	* Makefile.in (DOS_gnulib_comp.m4): Renamed from DOS-gnulib-comp.m4,
+	for portability to POSIX make.  Reported by Bruno Haible.
+	(sync-from-gnulib): Copy gl-comp.m4 (if present) back to
+	gnulib-comp.m4 before running gnulib-tool, to prevent old gnulib
+	files from accumulating as garbage.  Also reported by Bruno Haible.
+
 2011-01-27  Paul Eggert  <eggert@cs.ucla.edu>
 
 	fix two m4/gnulib-*.m4 file names that clashed under MS-DOS

=== modified file 'Makefile.in'
--- Makefile.in	2011-01-27 07:24:57 +0000
+++ Makefile.in	2011-01-28 00:25:24 +0000
@@ -325,7 +325,7 @@
 	git clone git://git.savannah.gnu.org/gnulib.git $@
 
 # A shorter name that satisfies MS-DOS 8+3 constraints.
-DOS-gnulib-comp.m4 = gl-comp.m4
+DOS_gnulib_comp.m4 = gl-comp.m4
 
 # Update modules from gnulib, for maintainers, who should have it in
 # $(gnulib_srcdir) (relative to $(srcdir) and should have build tools
@@ -334,10 +334,11 @@
 GNULIB_TOOL_FLAGS = \
  --import --no-changelog --no-vc-files --makefile-name=gnulib.mk
 sync-from-gnulib: $(gnulib_srcdir)
+	-cd $(srcdir)/m4 && cp $(DOS_gnulib_comp.m4) gnulib-comp.m4
 	cd $(srcdir) && \
 	  $(gnulib_srcdir)/gnulib-tool $(GNULIB_TOOL_FLAGS) $(GNULIB_MODULES)
 	cd $(srcdir)/m4 && rm gnulib-cache.m4 warn-on-use.m4
-	cd $(srcdir)/m4 && mv gnulib-comp.m4 $(DOS-gnulib-comp.m4)
+	cd $(srcdir)/m4 && mv gnulib-comp.m4 $(DOS_gnulib_comp.m4)
 	cp $(gnulib_srcdir)/build-aux/texinfo.tex $(srcdir)/doc/misc
 	cp \
 	  $(gnulib_srcdir)/build-aux/config.sub \
@@ -410,7 +411,7 @@
 $(srcdir)/configure: $(AUTOCONF_INPUTS)
 	cd ${srcdir} && autoconf
 
-ACLOCAL_INPUTS = @MAINT@ $(srcdir)/m4/$(DOS-gnulib-comp.m4)
+ACLOCAL_INPUTS = @MAINT@ $(srcdir)/m4/$(DOS_gnulib_comp.m4)
 $(srcdir)/aclocal.m4: $(ACLOCAL_INPUTS)
 	cd $(srcdir) && aclocal -I m4