updating macuvs.h?

updating macuvs.h?

[Stephen Leake]

I just did 'git pull' on master, and recompiled, on Windows with msys2
mingw64. Now git status shows 'src/macuvs.h' as changed. It has DOS line
endings instead of Unix; no other changes.

I assume we only want Unix line endings committed to git?

In which case, the makefile should run dos2unix (which is available in
mingw64).

-- 
-- Stephe

Re: updating macuvs.h?

[Paul Eggert]

On 01/13/2015 12:58 PM, Stephen Leake wrote:
> I assume we only want Unix line endings committed to git?

Yes, with a few exceptions; please see the file .gitattributes for the 
list of exceptions.

> In which case, the makefile should run dos2unix (which is available in
> mingw64).

No, it sounds like we need to fix the git configuration (perhaps by 
adding more lines to .gitattributes?) so that your problem doesn't 
recur.  It's better to not let CRLF problems leak into the makefiles.

Re: updating macuvs.h?

[Eli Zaretskii]

> From: Stephen Leake <stephen_leake@stephe-leake.org>
> Date: Tue, 13 Jan 2015 14:58:58 -0600
> 
> I just did 'git pull' on master, and recompiled, on Windows with msys2
> mingw64. Now git status shows 'src/macuvs.h' as changed. It has DOS line
> endings instead of Unix; no other changes.

This most probably means some of the tools involved in producing that
file are not MSYS programs, but native Windows (MinGW) programs.  MSYS
programs behave Posix-like and produce Unix-style EOLs.  Finds those
tools and replace them by their MSYS namesakes, and the problem will
go away.

> I assume we only want Unix line endings committed to git?
> 
> In which case, the makefile should run dos2unix (which is available in
> mingw64).

No, what you see shouldn't happen in the first place.

Re: updating macuvs.h?

[Stephen Leake]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stephen Leake <stephen_leake@stephe-leake.org>
>> Date: Tue, 13 Jan 2015 14:58:58 -0600
>> 
>> I just did 'git pull' on master, and recompiled, on Windows with msys2
>> mingw64. Now git status shows 'src/macuvs.h' as changed. It has DOS line
>> endings instead of Unix; no other changes.
>
> This most probably means some of the tools involved in producing that
> file are not MSYS programs, but native Windows (MinGW) programs.  MSYS
> programs behave Posix-like and produce Unix-style EOLs.  Finds those
> tools and replace them by their MSYS namesakes, and the problem will
> go away.

I think this is the makefile line that produces that file (from
admin/unidata/Makefile):

${top_srcdir}/src/macuvs.h: ${srcdir}/uvs.el ${srcdir}/IVD_Sequences.txt | \
  ${srcdir}/uvs.elc
	$(AM_V_GEN)${emacs} -L ${srcdir} -l uvs \
	  --eval '(uvs-print-table-ivd (unmsys--file-name "${srcdir}/IVD_Sequences.txt") "Adobe-Japan1")' \
	  > $@


This runs Emacs, which is a native MinGW program at this point. I don't
think we want to build an MSYS version just for this :).

That invokes this fuction (in admin/unidata/uvs.el):

(defun uvs-print-table-ivd (filename collection-id
				     &optional sequence-id-to-glyph-func)
  "Print a C array definition of a UVS table for IVD Sequences.
FILENAME specifies the IVD Sequences file.  COLLECTION-ID is a
string specifying the identifier of the collection to
extract (e.g., \"Adobe-Japan1\").  SEQUENCE-ID-TO-GLYPH-FUNC is a
function to convert an identifier string of the sequence to a
glyph number, and nil means to convert \"CID\\+[0-9]+\" to the
corresponding number."
  (or sequence-id-to-glyph-func
      (setq sequence-id-to-glyph-func
	    (lambda (sequence-id)
	      (string-match "\\`CID\\+\\([[:digit:]]+\\)\\'" sequence-id)
	      (string-to-number (match-string 1 sequence-id)))))
  (let ((uvs-alist
	 (with-temp-buffer
	   (insert-file-contents filename)
	   (uvs-alist-from-ivd collection-id
			       sequence-id-to-glyph-func))))
    (princ "/* Automatically generated by uvs.el.  */\n")
    (princ
     (format "static const unsigned char mac_uvs_table_%s_bytes[] =\n  {\n"
	     (replace-regexp-in-string "[^_[:alnum:]]" "_"
				       (downcase collection-id))))
    (with-temp-buffer
      (set-buffer-multibyte nil)
      (uvs-insert-alist-as-bytes uvs-alist)
      (uvs-dump))
    (princ "  };\n")))


`uvs-dump' uses `terpri' to insert the newline; that inserts "\n". So this
needs a `set-buffer-file-coding-system'?

I don't see where the buffer is actually written to the file.


On the other hand, why has no other MinGW builder seen this before?

-- 
-- Stephe

Re: updating macuvs.h?

[Eli Zaretskii]

> From: Stephen Leake <stephen_leake@stephe-leake.org>
> Date: Wed, 14 Jan 2015 14:43:46 -0600
> 
> I think this is the makefile line that produces that file (from
> admin/unidata/Makefile):
> 
> ${top_srcdir}/src/macuvs.h: ${srcdir}/uvs.el ${srcdir}/IVD_Sequences.txt | \
>   ${srcdir}/uvs.elc
> 	$(AM_V_GEN)${emacs} -L ${srcdir} -l uvs \
> 	  --eval '(uvs-print-table-ivd (unmsys--file-name "${srcdir}/IVD_Sequences.txt") "Adobe-Japan1")' \
> 	  > $@
> 
> 
> This runs Emacs, which is a native MinGW program at this point. I don't
> think we want to build an MSYS version just for this :).
> 
> That invokes this fuction (in admin/unidata/uvs.el):

Change the EOL conversion of the buffer where this is created to
-unix, and the problem should be solved.

> On the other hand, why has no other MinGW builder seen this before?

Because this file normally shouldn't be rebuilt, I think.

Re: updating macuvs.h?

[Eli Zaretskii]

> Date: Thu, 15 Jan 2015 05:48:41 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> > From: Stephen Leake <stephen_leake@stephe-leake.org>
> > Date: Wed, 14 Jan 2015 14:43:46 -0600
> > 
> > I think this is the makefile line that produces that file (from
> > admin/unidata/Makefile):
> > 
> > ${top_srcdir}/src/macuvs.h: ${srcdir}/uvs.el ${srcdir}/IVD_Sequences.txt | \
> >   ${srcdir}/uvs.elc
> > 	$(AM_V_GEN)${emacs} -L ${srcdir} -l uvs \
> > 	  --eval '(uvs-print-table-ivd (unmsys--file-name "${srcdir}/IVD_Sequences.txt") "Adobe-Japan1")' \
> > 	  > $@
> > 
> > 
> > This runs Emacs, which is a native MinGW program at this point. I don't
> > think we want to build an MSYS version just for this :).
> > 
> > That invokes this fuction (in admin/unidata/uvs.el):
> 
> Change the EOL conversion of the buffer where this is created to
> -unix, and the problem should be solved.

Actually, this wouldn't have worked, since uvs.el doesn't save text
from a buffer, but instead uses princ etc. to write directly to
stdout.

So I added a new feature that allows switching a standard stream to
binary mode, and this problem should now be solved, please test.

Re: updating macuvs.h?

[Stephen Leake]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Thu, 15 Jan 2015 05:48:41 +0200
>> From: Eli Zaretskii <eliz@gnu.org>
>> Cc: emacs-devel@gnu.org
>> 
>> > From: Stephen Leake <stephen_leake@stephe-leake.org>
>> > Date: Wed, 14 Jan 2015 14:43:46 -0600
>> > 
>> > I think this is the makefile line that produces that file (from
>> > admin/unidata/Makefile):
>> > 
>> > ${top_srcdir}/src/macuvs.h: ${srcdir}/uvs.el ${srcdir}/IVD_Sequences.txt | \
>> >   ${srcdir}/uvs.elc
>> > 	$(AM_V_GEN)${emacs} -L ${srcdir} -l uvs \
>> > 	  --eval '(uvs-print-table-ivd (unmsys--file-name "${srcdir}/IVD_Sequences.txt") "Adobe-Japan1")' \
>> > 	  > $@
>> > 
>> > 
>> > This runs Emacs, which is a native MinGW program at this point. I don't
>> > think we want to build an MSYS version just for this :).
>> > 
>> > That invokes this fuction (in admin/unidata/uvs.el):
>> 
>> Change the EOL conversion of the buffer where this is created to
>> -unix, and the problem should be solved.
>
> Actually, this wouldn't have worked, since uvs.el doesn't save text
> from a buffer, but instead uses princ etc. to write directly to
> stdout.
>
> So I added a new feature that allows switching a standard stream to
> binary mode, and this problem should now be solved, please test.


This 'git pull' resulted in re-running configure; I suspect src/macuvs.h
will always be rebuilt when that happens.


It now generates Unix line endings, but git complains about the
permissions:

stephe@takver$ git diff src/macuvs.h
diff --git a/src/macuvs.h b/src/macuvs.h
old mode 100644
new mode 100755

My MinGW64 shell has umask set properly:

$ umask
0022
$ umask -S
u=rwx,g=rx,o=rx

So this is being overridden by Emacs.

-- 
-- Stephe

Re: updating macuvs.h?

[Eli Zaretskii]

> From: Stephen Leake <stephen_leake@stephe-leake.org>
> Date: Fri, 16 Jan 2015 13:57:26 -0600
> 
> This 'git pull' resulted in re-running configure; I suspect src/macuvs.h
> will always be rebuilt when that happens.

No, running configure doesn't trigger rebuilding that file, at least
not for me.

> It now generates Unix line endings, but git complains about the
> permissions:
> 
> stephe@takver$ git diff src/macuvs.h
> diff --git a/src/macuvs.h b/src/macuvs.h
> old mode 100644
> new mode 100755

Doesn't happen to me.

> My MinGW64 shell has umask set properly:
> 
> $ umask
> 0022
> $ umask -S
> u=rwx,g=rx,o=rx
> 
> So this is being overridden by Emacs.

The native build of Emacs has no way of setting the execute bit,
because it doesn't exist in the filesystem, as far as Emacs is
concerned.  So some other factor is at work here.

My MSYS umask is 0000, FWIW.