[NT] Problem with `man' function

[NT] Problem with `man' function

[David Abrahams]

With Cygwin installed and in my PATH, `M-x man' used to work fine for
me.  In the past year, it has stopped working; all I get is "error in
process sentinel. Can't find the xxx manpage."  I was able to work
around the problem either by setting  shell-file-name to 

  "sh" 

or, guessing that it was a CMD.EXE line length issue, by shortening
Man-filter-list to

  (("sed" "" "-e '/^[^A-^Z][^A-^Z]*$/d'" "-e '/^[[789]/s///g'" ))

I don't know what the best fix might be.  Can anyone tell me the
implications of setting shell-file-name to "sh?"

Thanks,
-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com

Re: [NT] Problem with `man' function

[Eli Zaretskii]

> From: David Abrahams <dave@boost-consulting.com>
> Date: Sun, 23 Oct 2005 08:02:56 -0400
> 
> With Cygwin installed and in my PATH, `M-x man' used to work fine for
> me.  In the past year, it has stopped working; all I get is "error in
> process sentinel. Can't find the xxx manpage."  I was able to work
> around the problem either by setting  shell-file-name to 
> 
>   "sh" 
> 
> or, guessing that it was a CMD.EXE line length issue, by shortening
> Man-filter-list to
> 
>   (("sed" "" "-e '/^[^A-^Z][^A-^Z]*$/d'" "-e '/^[[789]/s///g'" ))

I don't think this has anything to do with CMD command-line
limitations: the filter script is only about 700 characters, much less
than the CMD's 2KB command limitation.  Quoting arguments with '..',
however (see above) might very well be the reason for the failure when
CMD is the shell, because Windows programs in general don't understand
'..' quoting, and CMD, unlike Bash, doesn't remove the '' from
command-line arguments.

> I don't know what the best fix might be.

It would be best if you debugged the problem and told us precisely
what the problem was; your description sounds like you aren't really
sure.  It could as well be a Cygwin bug, or some change in Emacs that
exposes a Cygwin bug.

> Can anyone tell me the implications of setting shell-file-name to
> "sh?"

All your shell commands will be run via (presumably Cygwin port of)
Bash, so native CMD commands will have to be preceded by "cmd /c".

Re: [NT] Problem with `man' function

[Richard M. Stallman]

    With Cygwin installed and in my PATH, `M-x man' used to work fine for
    me.  In the past year, it has stopped working; all I get is "error in
    process sentinel. Can't find the xxx manpage."

Could you possibly try debugging it and finding out what is really
wrong?

    I don't know what the best fix might be.

Nobody can tell what the best fix is
without first finding out what is really wrong.

Re: [NT] Problem with `man' function

[David Abrahams]

"Richard M. Stallman" <rms@gnu.org> writes:

>     With Cygwin installed and in my PATH, `M-x man' used to work fine for
>     me.  In the past year, it has stopped working; all I get is "error in
>     process sentinel. Can't find the xxx manpage."
>
> Could you possibly try debugging it and finding out what is really
> wrong?
>
>     I don't know what the best fix might be.
>
> Nobody can tell what the best fix is
> without first finding out what is really wrong.

I can tell you that if you remove the awk command at the end of the
filter chain you still get something that looks like a man page.
Otherwise, you get something that looks like 

  0 (00, 0x0) = ?\C-@

And the process sentinel, understandably, has a hard time parsing it.
I don't know enough about awk nor about the NT shell command processor
to get much further with this one.

HTH,
-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com

Re: [NT] Problem with `man' function

[Richard M. Stallman]

    > Nobody can tell what the best fix is
    > without first finding out what is really wrong.

    I can tell you that if you remove the awk command at the end of the
    filter chain you still get something that looks like a man page.
    Otherwise, you get something that looks like 

      0 (00, 0x0) = ?\C-@

    And the process sentinel, understandably, has a hard time parsing it.
    I don't know enough about awk nor about the NT shell command processor
    to get much further with this one.

This might be enough to enable the people interested in Windows
to fix it.

Re: [NT] Problem with `man' function

[Eli Zaretskii]

> From: "Richard M. Stallman" <rms@gnu.org>
> Date: Wed, 26 Oct 2005 21:29:07 -0400
> Cc: emacs-devel@gnu.org
> 
>     > Nobody can tell what the best fix is
>     > without first finding out what is really wrong.
> 
>     I can tell you that if you remove the awk command at the end of the
>     filter chain you still get something that looks like a man page.
>     Otherwise, you get something that looks like 
> 
>       0 (00, 0x0) = ?\C-@
> 
>     And the process sentinel, understandably, has a hard time parsing it.
>     I don't know enough about awk nor about the NT shell command processor
>     to get much further with this one.
> 
> This might be enough to enable the people interested in Windows
> to fix it.

Not for me, it isn't.  Sorry.

To try to debug and fix it, I need to know:

  (1) What shell is being used to run the commands in the pipeline
      that produces the man page;

  (2) What ports of gawk and Sed are used (presumably Cygwin, but
      that's a guess).

Re: [NT] Problem with `man' function

[David Abrahams]

Eli Zaretskii <eliz@gnu.org> writes:

> To try to debug and fix it, I need to know:
>
>   (1) What shell is being used to run the commands in the pipeline
>       that produces the man page;

The regular Windows XP shell (CMD)

>   (2) What ports of gawk and Sed are used (presumably Cygwin, but
>       that's a guess).

Cygwin
GNU Awk 3.1.5
GNU sed version 4.1.4


-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com

Re: [NT] Problem with `man' function

[Eli Zaretskii]

> From: David Abrahams <dave@boost-consulting.com>
> Cc: rms@gnu.org,  emacs-devel@gnu.org
> Date: Thu, 27 Oct 2005 07:00:27 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > To try to debug and fix it, I need to know:
> >
> >   (1) What shell is being used to run the commands in the pipeline
> >       that produces the man page;
> 
> The regular Windows XP shell (CMD)
> 
> >   (2) What ports of gawk and Sed are used (presumably Cygwin, but
> >       that's a guess).
> 
> Cygwin
> GNU Awk 3.1.5
> GNU sed version 4.1.4

I tried to debug this, but I'm sorry to say that I cannot reproduce
exactly the behavior you describe.  "M-x man" indeed fails for me in
the Windows port, but the error message displayed by Emacs is
different:

    sed: -e expression #1, char 1: unknown command: `''
    awk: '
    awk: ^ invalid char ''' in expression

    process exited abnormally with code 1

The messages from Sed and Awk say quite unequivocally that the problem
is with the quoting style: Windows shells and programs don't support
the '..' quoting that man.el uses.

I tried to replace '..' quoting with ".." quoting, but then bumped
into other bugs, this time in Sed.  It sounds like the GnuWin32 port
of Sed is hopelessly broken.  Or it could be a bug with cmdproxy.exe,
I really don't have time to dig into that.

Perhaps with the Cygwin ports you have man.el will work if you replace
'..' quoting with ".." quoting in the Sed and Awk scripts used by
man.el.  Could you perhaps try that?

Failing that, I suggest to use woman.el.

Re: [NT] Problem with `man' function

[Dave Abrahams]

Eli Zaretskii <eliz <at> gnu.org> writes:

> I tried to debug this, but I'm sorry to say that I cannot reproduce
> exactly the behavior you describe.  

Interesting.  Still happens for me with the latest CVS of emacs and recently
updated Cygwin.  And it's clearly the awk command that's causing it.  If I
delete the awk command, all is well.

> "M-x man" indeed fails for me in
> the Windows port, but the error message displayed by Emacs is
> different:
> 
>     sed: -e expression #1, char 1: unknown command: `''
>     awk: '
>     awk: ^ invalid char ''' in expression
> 
>     process exited abnormally with code 1
> 
> The messages from Sed and Awk say quite unequivocally that the problem
> is with the quoting style: Windows shells and programs don't support
> the '..' quoting that man.el uses.

Using the other quoting made no difference whatsoever for me.

> I tried to replace '..' quoting with ".." quoting, but then bumped
> into other bugs, this time in Sed.  It sounds like the GnuWin32 port
> of Sed is hopelessly broken.  Or it could be a bug with cmdproxy.exe,
> I really don't have time to dig into that.
> 
> Perhaps with the Cygwin ports you have man.el will work if you replace
> '..' quoting with ".." quoting in the Sed and Awk scripts used by
> man.el.  Could you perhaps try that?

Tried it, to no avail.

> Failing that, I suggest to use woman.el.

Aww, this is discouraging.  Can't you suggest a place to breakpoint so we can
look at what's coming out of awk?

--
David Abrahams
Boost Consulting
http://www.boost-consulting.com

Re: [NT] Problem with `man' function

[Eli Zaretskii]

> From: Dave Abrahams <dave@boost-consulting.com>
> Date: Wed, 10 May 2006 20:54:16 +0000 (UTC)
> 
> >     sed: -e expression #1, char 1: unknown command: `''
> >     awk: '
> >     awk: ^ invalid char ''' in expression
> > 
> >     process exited abnormally with code 1
> > 
> > The messages from Sed and Awk say quite unequivocally that the problem
> > is with the quoting style: Windows shells and programs don't support
> > the '..' quoting that man.el uses.
> 
> Using the other quoting made no difference whatsoever for me.
> 
> > I tried to replace '..' quoting with ".." quoting, but then bumped
> > into other bugs, this time in Sed.  It sounds like the GnuWin32 port
> > of Sed is hopelessly broken.  Or it could be a bug with cmdproxy.exe,
> > I really don't have time to dig into that.
> > 
> > Perhaps with the Cygwin ports you have man.el will work if you replace
> > '..' quoting with ".." quoting in the Sed and Awk scripts used by
> > man.el.  Could you perhaps try that?
> 
> Tried it, to no avail.

Please show the modified portion in man.el with this change.  Perhaps
you didn't get all the escaping right.

> > Failing that, I suggest to use woman.el.
> 
> Aww, this is discouraging.

Why?  There's nothing man.el can do that woman.el cannot.

> Can't you suggest a place to breakpoint so we can look at what's
> coming out of awk?

>From the above error message it's clear that nothing comes out of Awk,
since it barfs because of a syntax error.

But if you do manage to run Awk without an error message, just add a
`message' line that displays its output after it returns.

Re: [NT] Problem with `man' function

[David Abrahams]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Dave Abrahams <dave@boost-consulting.com>
>> Date: Wed, 10 May 2006 20:54:16 +0000 (UTC)
>> 
>> >     sed: -e expression #1, char 1: unknown command: `''
>> >     awk: '
>> >     awk: ^ invalid char ''' in expression
>> > 
>> >     process exited abnormally with code 1
>> > 
>> > The messages from Sed and Awk say quite unequivocally that the problem
>> > is with the quoting style: Windows shells and programs don't support
>> > the '..' quoting that man.el uses.
>> 
>> Using the other quoting made no difference whatsoever for me.
>> 
>> > I tried to replace '..' quoting with ".." quoting, but then bumped
>> > into other bugs, this time in Sed.  It sounds like the GnuWin32 port
>> > of Sed is hopelessly broken.  Or it could be a bug with cmdproxy.exe,
>> > I really don't have time to dig into that.
>> > 
>> > Perhaps with the Cygwin ports you have man.el will work if you replace
>> > '..' quoting with ".." quoting in the Sed and Awk scripts used by
>> > man.el.  Could you perhaps try that?
>> 
>> Tried it, to no avail.
>
> Please show the modified portion in man.el with this change.  Perhaps
> you didn't get all the escaping right.

I used the emacs customization interface, and so avoided all that.  I
just deleted the awk part.

>> > Failing that, I suggest to use woman.el.
>> 
>> Aww, this is discouraging.
>
> Why?  There's nothing man.el can do that woman.el cannot.

I know that song from somewhere... :)

>> Can't you suggest a place to breakpoint so we can look at what's
>> coming out of awk?
>
>>From the above error message it's clear that nothing comes out of Awk,
> since it barfs because of a syntax error.

Oh.

> But if you do manage to run Awk without an error message, just add a
> `message' line that displays its output after it returns.

Okay.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com

Re: [NT] Problem with `man' function

[David Abrahams]

David Abrahams <dave@boost-consulting.com> writes:

>> But if you do manage to run Awk without an error message, just add a
>> `message' line that displays its output after it returns.
>
> Okay.

Problem(s) diagnosed:

1. awk.exe is just a cygwin symlink for me, and since I'm not running
   emacs under cygwin... boom!

2. quoting for awk needs to use double quotes, not single quotes for
   some reason

3. CMD doesn't cope, no way, no how, with newlines in the command line
   (at least, not in any way I've found)

Recipe for a fix in my environment, in Man-filter-list:

a. replace "awk" with "gawk"
b. remove all newlines from the gawk arguments
c. backslash-escape existing double quotes in the gawk arguments
d. replace single quotes in the gawk arguments with double quotes


(setq Man-filter-list (quote (("sed" "" "-e '/^[\x01-\x1a][\x01-\x1a]*$/d'" "-e '/^[[789]/s///g'" "-e '/Reformatting page.  Wait/d'" "-e '/Reformatting entry.  Wait/d'" "-e '/^[ 	]*Hewlett-Packard[ 	]Company[ 	]*-[ 	][0-9]*[ 	]-/d'" "-e '/^[ 	]*Hewlett-Packard[ 	]*-[ 	][0-9]*[ 	]-.*$/d'" "-e '/^[ 	][ 	]*-[ 	][0-9]*[ 	]-[ 	]*Formatted:.*[0-9]$/d'" "-e '/^[ 	]*Page[ 	][0-9]*.*(printed[ 	][0-9\\/]*)$/d'" "-e '/^Printed[ 	][0-9].*[0-9]$/d'" "-e '/^[ 	]*X[ 	]Version[ 	]1[01].*Release[ 	][0-9]/d'" "-e '/^[A-Za-z].*Last[ 	]change:/d'" "-e '/^Sun[ 	]Release[ 	][0-9].*[0-9]$/d'" "-e '/[ 	]*Copyright [0-9]* UNIX System Laboratories, Inc.$/d'" "-e '/^[ 	]*Rev\\..*Page [0-9][0-9]*$/d'") ("gawk" "\"" "BEGIN { blankline=0; anonblank=0; }" "/^$/ { if (anonblank==0) next; }" "{ anonblank=1; }" "/^$/ { blankline++; next; }" "{ if (blankline>0) { print \\\"\\\"; blankline=0; } print $0; }" "\"")))

Regards,

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com

Re: [NT] Problem with `man' function

[Eli Zaretskii]

> From: David Abrahams <dave@boost-consulting.com>
> Cc: emacs-devel@gnu.org
> Date: Thu, 11 May 2006 07:44:22 -0400
> 
> Problem(s) diagnosed:

Thanks.

> 2. quoting for awk needs to use double quotes, not single quotes for
>    some reason

What port of awk do you use?  Is it a Cygwin port or a native Windows
port?

> 3. CMD doesn't cope, no way, no how, with newlines in the command line
>    (at least, not in any way I've found)

Yes, it's a known problem.

> Recipe for a fix in my environment, in Man-filter-list:
> 
> a. replace "awk" with "gawk"
> b. remove all newlines from the gawk arguments
> c. backslash-escape existing double quotes in the gawk arguments
> d. replace single quotes in the gawk arguments with double quotes

Thanks, I will try to incorporate this into man.el.

Re: [NT] Problem with `man' function

[David Abrahams]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: David Abrahams <dave@boost-consulting.com>
>> Cc: emacs-devel@gnu.org
>> Date: Thu, 11 May 2006 07:44:22 -0400
>> 
>> Problem(s) diagnosed:
>
> Thanks.
>
>> 2. quoting for awk needs to use double quotes, not single quotes for
>>    some reason
>
> What port of awk do you use?  Is it a Cygwin port or a native Windows
> port?

Cygwin.

>> 3. CMD doesn't cope, no way, no how, with newlines in the command line
>>    (at least, not in any way I've found)
>
> Yes, it's a known problem.
>
>> Recipe for a fix in my environment, in Man-filter-list:
>> 
>> a. replace "awk" with "gawk"
>> b. remove all newlines from the gawk arguments
>> c. backslash-escape existing double quotes in the gawk arguments
>> d. replace single quotes in the gawk arguments with double quotes
>
> Thanks, I will try to incorporate this into man.el.

Thank you, very much.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com