grep-find with Polish letters in Windows

grep-find with Polish letters in Windows

[Andrzej Skiba]

[-- Attachment #1: Type: text/plain, Size: 1411 bytes --]

Hi,
I have the following function to search my projects with grep-find:

(defun as/grep-project (project pattern)
  (interactive "sProject: \nsPattern: ")
    (grep-find (concat
                   "/usr/bin/find /cygdrive/c/projects/"
                   project
                   " -type f "
                   " -not -name \"*.svn-base\" "
                   "-and -not -name \"*.tmp\" "
                   "-and -not -name \"*.log\" -print0 "
                   "| xargs -0 -e grep -U -n -s -F \""
                   pattern
                   "\"")))

All works great until I try to search for a word with Polish letters (such
as ą, ś, ć, ł, ń etc.). The files are all utf-8. When I run the command
searching for string "Usuń" in project test I get the following output in
the grep buffer:

/usr/bin/find /cygdrive/c/projects/test -type f -not -name "*.svn-base" -and
-not -name "*.tmp" -and -not -name "*.log" -print0 | xargs -0 -e grep -U -n
-s -F "Usuń"
/usr/bin/bash: /usr/bin/find /cygdrive/c/projects/test -type f -not -name
"*.svn-base" -and -not -name "*.tmp" -and -not -name "*.log" -print0 | xargs
-0 -e grep -U -n -s -F "Usuń": No such file or directory

It runs fine with any input without Polish characters.

The find command works fine in a regular window shell as well as cygwin
bash.

Any ideas?

Thanks for any help, regards,

Andrzej Skiba

[-- Attachment #2: Type: text/html, Size: 2151 bytes --]

Re: grep-find with Polish letters in Windows

[Eli Zaretskii]

> Date: Tue, 14 Sep 2010 13:02:30 +0200
> From: Andrzej Skiba <andskiba@gmail.com>
> 
> (defun as/grep-project (project pattern)
>   (interactive "sProject: \nsPattern: ")
>     (grep-find (concat
>                    "/usr/bin/find /cygdrive/c/projects/"
>                    project
>                    " -type f "
>                    " -not -name \"*.svn-base\" "
>                    "-and -not -name \"*.tmp\" "
>                    "-and -not -name \"*.log\" -print0 "
>                    "| xargs -0 -e grep -U -n -s -F \""
>                    pattern
>                    "\"")))
> 
> All works great until I try to search for a word with Polish letters (such
> as ą, ś, ć, ł, ń etc.). The files are all utf-8. When I run the command
> searching for string "Usuń" in project test I get the following output in
> the grep buffer:
> 
> /usr/bin/find /cygdrive/c/projects/test -type f -not -name "*.svn-base" -and
> -not -name "*.tmp" -and -not -name "*.log" -print0 | xargs -0 -e grep -U -n
> -s -F "Usuń"
> /usr/bin/bash: /usr/bin/find /cygdrive/c/projects/test -type f -not -name
> "*.svn-base" -and -not -name "*.tmp" -and -not -name "*.log" -print0 | xargs
> -0 -e grep -U -n -s -F "Usuń": No such file or directory
> 
> It runs fine with any input without Polish characters.

You seem to be using the native build of Emacs in conjunction with
Cygwin tools (Grep, Bash, etc.).  If so, this is asking for trouble,
because there are subtle incompatibilities between Cygwin programs and
native Windows programs.  I/O encoding is one of these areas: whereas
latest Cygwin versions use UTF-8, native Windows programs use the
current Windows codepage.  The native build of Emacs cannot encode
command lines it passes to programs in anything but the current
codepage, which is no good for you if your files are encoded in UTF-8.

I suggest to use the Cygwin build of Emacs instead.

> The find command works fine in a regular window shell as well as cygwin
> bash.

What is the "regular window shell"?  If it's CMD, then I don't see how
it could work, since CMD does not support UTF-8 keyboard input.
Perhaps the Cygwin port of Grep transparently converts keyboard input
into UTF-8 or something.

Re: grep-find with Polish letters in Windows

[Andrzej Skiba]

[-- Attachment #1: Type: text/plain, Size: 2778 bytes --]

I switched to the cygwin build of emacs and everything is working as
expected so thank you for your help. I usually do my work on a Linux box,
but they gave me a Windows machine at work recently. It's been such a pain
to work with sometimes.

Thanks again,

Andrzej

PS. And yes CMD is what I meant by "regular windows shell" :)

On Tue, Sep 14, 2010 at 9:21 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Tue, 14 Sep 2010 13:02:30 +0200
> > From: Andrzej Skiba <andskiba@gmail.com>
> >
> > (defun as/grep-project (project pattern)
> >   (interactive "sProject: \nsPattern: ")
> >     (grep-find (concat
> >                    "/usr/bin/find /cygdrive/c/projects/"
> >                    project
> >                    " -type f "
> >                    " -not -name \"*.svn-base\" "
> >                    "-and -not -name \"*.tmp\" "
> >                    "-and -not -name \"*.log\" -print0 "
> >                    "| xargs -0 -e grep -U -n -s -F \""
> >                    pattern
> >                    "\"")))
> >
> > All works great until I try to search for a word with Polish letters
> (such
> > as ą, ś, ć, ł, ń etc.). The files are all utf-8. When I run the command
> > searching for string "Usuń" in project test I get the following output in
> > the grep buffer:
> >
> > /usr/bin/find /cygdrive/c/projects/test -type f -not -name "*.svn-base"
> -and
> > -not -name "*.tmp" -and -not -name "*.log" -print0 | xargs -0 -e grep -U
> -n
> > -s -F "Usuń"
> > /usr/bin/bash: /usr/bin/find /cygdrive/c/projects/test -type f -not -name
> > "*.svn-base" -and -not -name "*.tmp" -and -not -name "*.log" -print0 |
> xargs
> > -0 -e grep -U -n -s -F "Usuń": No such file or directory
> >
> > It runs fine with any input without Polish characters.
>
> You seem to be using the native build of Emacs in conjunction with
> Cygwin tools (Grep, Bash, etc.).  If so, this is asking for trouble,
> because there are subtle incompatibilities between Cygwin programs and
> native Windows programs.  I/O encoding is one of these areas: whereas
> latest Cygwin versions use UTF-8, native Windows programs use the
> current Windows codepage.  The native build of Emacs cannot encode
> command lines it passes to programs in anything but the current
> codepage, which is no good for you if your files are encoded in UTF-8.
>
> I suggest to use the Cygwin build of Emacs instead.
>
> > The find command works fine in a regular window shell as well as cygwin
> > bash.
>
> What is the "regular window shell"?  If it's CMD, then I don't see how
> it could work, since CMD does not support UTF-8 keyboard input.
> Perhaps the Cygwin port of Grep transparently converts keyboard input
> into UTF-8 or something.
>

[-- Attachment #2: Type: text/html, Size: 3683 bytes --]