emacs in text mode terminal paste problem (SecureCRT / xterm )

emacs in text mode terminal paste problem (SecureCRT / xterm )

[emacsuser]

Hi all! 

I've experienced problems pasting apostrophes or bullets originating
from rich text documents like Word into an emacs running in text mode
in a SecureCRT window.  I run emacs 21.4.1 in text mode via ssh using
SecureCRT using xterm emulation.   

Problems pasting bullets from Word or Notepad into SecureCRT terminal window: 
------------------------------------------------------------------
When pasting text that includes bullet from word (that uses Windows
Symbol font), the paste generates an error in emacs;
       Scan error: "Unbalanced parentheses", 1790, 1

I have no idea what emacs function the bullet is causing to run there.

Problems pasting an apostrophe ' from Word into SecureCT terminal window:
-------------------------------------------------------------------------
An apostrophe that get pasted appears to invoke "Regexp I-search backward" somehow.
The search of course usually fails, because it can't find all of the
text that follows.

For example,  if I select and copy the following line in Word: 
    Bob's responsibilities include:   

And paste it into the securecrt window where emacs is running inside
of screen, emacs jumps the cursor backwards to where it finds the
first instance of the letter s followed by a space, and gives the
error:

    Failing regexp I-search backward: s responsibilities include: 

And the lines that follow the line that had the ' on it, end up
getting inserted into the middle of the rest of things depending on
where the search failed (usually after a word that ends in s followed
by a space).  It generally ends up looking like a dog's breakfast.

Interestingly, apostrophe's created, and copied from Notepad... no
problem.   Apostrophes from Word, pasted into Notepad, copied from
notepad and then into the terminal window still exhibit the issue.
It's friggin weird. 

Also interestingly, these issues don't occur when using ssh in a
cygwin windows with the TERM set to vt220.  Emacs -nw over ssh in
cygwin with the term set to cygwin is a complete catastrophe, but
that's an issue for another time.

Ideas?  Experiences?  Workarounds?

Bob 

Re: emacs in text mode terminal paste problem (SecureCRT / xterm )

[Niels Giesen]

emacsuser@invalid.org writes:

> Hi all! 
>
> I've experienced problems pasting apostrophes or bullets originating
> from rich text documents like Word into an emacs running in text mode
> in a SecureCRT window.  I run emacs 21.4.1 in text mode via ssh using
> SecureCRT using xterm emulation.   
>
> Problems pasting bullets from Word or Notepad into SecureCRT terminal window: 
> ------------------------------------------------------------------
> When pasting text that includes bullet from word (that uses Windows
> Symbol font), the paste generates an error in emacs;
>        Scan error: "Unbalanced parentheses", 1790, 1
>
> I have no idea what emacs function the bullet is causing to run there.
>
> Problems pasting an apostrophe ' from Word into SecureCT terminal window:
> -------------------------------------------------------------------------
> An apostrophe that get pasted appears to invoke "Regexp I-search backward" somehow.
> The search of course usually fails, because it can't find all of the
> text that follows.
>
> For example,  if I select and copy the following line in Word: 
>     Bob's responsibilities include:   
>
> And paste it into the securecrt window where emacs is running inside
> of screen, emacs jumps the cursor backwards to where it finds the
> first instance of the letter s followed by a space, and gives the
> error:
>
>     Failing regexp I-search backward: s responsibilities include: 
>
> And the lines that follow the line that had the ' on it, end up
> getting inserted into the middle of the rest of things depending on
> where the search failed (usually after a word that ends in s followed
> by a space).  It generally ends up looking like a dog's breakfast.
>
> Interestingly, apostrophe's created, and copied from Notepad... no
> problem.   Apostrophes from Word, pasted into Notepad, copied from
> notepad and then into the terminal window still exhibit the issue.
> It's friggin weird. 
>
> Also interestingly, these issues don't occur when using ssh in a
> cygwin windows with the TERM set to vt220.  Emacs -nw over ssh in
> cygwin with the term set to cygwin is a complete catastrophe, but
> that's an issue for another time.
>
> Ideas?  Experiences?  Workarounds?

Just a hunch:

The apostrophe generated by word might be a true apostrophe somewhere
higher up the Unicode ladder. Apostrophes, single opening and ending
quotes and the sign for minutes are all input using the ' key on your
keyboard but changed on the fly to their respective typographical
forms. Even if the glyphs look the same (in case of a bad font), the
character codes are different.

If that is the case, try and paste other Unicode letters, to generalize
the issue. It seems somewhere things get mangled (maybe control chars
being sent to emacs). What happens if you paste it into a screen window
not running emacs (e.g. pasting onto the command line)?

Maybe a workaround is to convert the word document to plain text using
antiword first. I think this is/can be present on cygwin.

Another workaround, if possible (?), might be to turn off automatic
apostrophe conversion in word.

-- 
http://niels.kicks-ass.org

Re: emacs in text mode terminal paste problem (SecureCRT / xterm )

[Peter Dyballa]

Am 23.11.2008 um 00:14 schrieb emacsuser@invalid.org:

> An apostrophe that get pasted appears to invoke "Regexp I-search  
> backward" somehow.
> The search of course usually fails, because it can't find all of the
> text that follows.


The so-called apostrophes from MS Word could be single quote  
characters from the area UTF+201x. Maybe it works to C-q in some  
(*scratch*) buffer and then paste just that "apostrophe." You might  
see the bytes that would be actually inserted. The three bytes of  
UTF-8 representation (E2 80 99 for ’, RIGHT SINGLE QUOTATION MARK)  
can be interpreted as this: E2 is in 7-bit idiom C-r (single 8th bit  
set + remainder r), the other two bytes can be any 7-bit rubbish.

GNU Emacs offers some "translations" for X selections – lookup info!  
X11 also has some configuration options. A better option is to  
upgrade to GNU Emacs 22.3.

--
Greetings

   Pete

Isn't vi that text editor with two modes... one that beeps and one  
that corrupts your file?
				– Dan Jacobson, on comp.os.linux.advocacy

Re: emacs in text mode terminal paste problem (SecureCRT / xterm )

[Xah Lee]

Some general experience regard your problem.

First of all, i like to note that SecureCRT is the ssh terminal of my
choice in Windows platform, during the time when i was active in IT
industry working on web stores, where Windows NT is used as company
desktop, and Solaris are servers. This is during 1998 to 2002, i
recall as a sys admin i ordered this software for all company to use.
However, i haven't used Windows since about 2005.

The general problem of copying and pasting thru different apps to
server involves quite some complexities. In your example, when you
copy a text, it first goes thru some munching from Window's system
wide clipboard to your SecureCRT app, then from there thru telnet/ssh
protocol. There in fact involves multiple instances of translations
(at least 2 but i think more).

When you copy a text from a Windows app to another, the copied text is
in general not plain text, but involves possibly multiple versions
such as rich text version and plain text version, and when you paste
in another app, which version is pasted depends on the OS's clipboard
system and the app. (for example, on the mac, you can copy text from
Safari browser into Mac's TextEdit, the result is rich text that
retains somewhat of the font and formatting (but not exactly), and
when you paste into Apple Mail (with preference set to support html
mail), the result also retains the formatting/font somewhat, but note
that it has internally turned into html. And when you paste into, for
example, Carbon Emacs or Aquamacs Emacs, you get plain text.) So, the
point is that there are a lot conversions taking place. This
conversions are not trivial. It depends on the OS's system of
clipboard, and depends on what app you pasting it into.

Also, line ending and text encoding may also have multated. Some app
automatically convert line endings or text encoding.

And when the copy/pasting involves telnet/ssh, it goes into another
conversion because in general telnet as far as i knew in 2000, it
basically support transmission using ascii.

So, the only thing i can suggest is that given the above info, try to
copy/paste so that it it goes thru minimal number of conversions, and
find out where it went wrong, which character got wrong. Terminal app
or telnet is especially problematic. Many of them simply don't support
unicode or non-ascii chars, and some you need to experiment with its
usually cryptic config/preference/options a lot in order to get it to
work with a particular encoding.

looking at your problem, i'm pretty sure SecureCRT tried to turn your
unicode into some ascii with some control chars (esp sequences), of
which emacs thinks you are pressing Meta. I'm not actually sure you
can actually paste unicode thru terminal/telnet even today ...

... if you use another tunneling techonolgy, or some remote desktop
technology, it may bypass all these telnet/ssh problems.

  Xah
∑ http://xahlee.org/

☄

On Nov 22, 3:14 pm, emacsu...@invalid.org wrote:
> Hi all!
>
> I've experienced problems pasting apostrophes or bullets originating
> from rich text documents like Word into an emacs running in text mode
> in a SecureCRT window.  I run emacs 21.4.1 in text mode via ssh using
> SecureCRT using xterm emulation.
>
> Problems pasting bullets from Word or Notepad into SecureCRT terminal window:
> ------------------------------------------------------------------
> When pasting text that includes bullet from word (that uses Windows
> Symbol font), the paste generates an error in emacs;
>        Scan error: "Unbalanced parentheses", 1790, 1
>
> I have no idea what emacs function the bullet is causing to run there.
>
> Problems pasting an apostrophe ' from Word into SecureCT terminal window:
> -------------------------------------------------------------------------
> An apostrophe that get pasted appears to invoke "Regexp I-search backward" somehow.
> The search of course usually fails, because it can't find all of the
> text that follows.
>
> For example,  if I select and copy the following line in Word:
>     Bob's responsibilities include:
>
> And paste it into the securecrt window where emacs is running inside
> of screen, emacs jumps the cursor backwards to where it finds the
> first instance of the letter s followed by a space, and gives the
> error:
>
>     Failing regexp I-search backward: s responsibilities include:
>
> And the lines that follow the line that had the ' on it, end up
> getting inserted into the middle of the rest of things depending on
> where the search failed (usually after a word that ends in s followed
> by a space).  It generally ends up looking like a dog's breakfast.
>
> Interestingly, apostrophe's created, and copied from Notepad... no
> problem.   Apostrophes from Word, pasted into Notepad, copied from
> notepad and then into the terminal window still exhibit the issue.
> It's friggin weird.
>
> Also interestingly, these issues don't occur when using ssh in a
> cygwin windows with the TERM set to vt220.  Emacs -nw over ssh in
> cygwin with the term set to cygwin is a complete catastrophe, but
> that's an issue for another time.
>
> Ideas?  Experiences?  Workarounds?
>
> Bob

Re: emacs in text mode terminal paste problem (SecureCRT / xterm )

[Phil Carmody]

Xah Lee <xahlee@gmail.com> writes:


Fuck. My killfile for gnu.emacs.help only has 2 entries:

(("from"
  ("Xah Lee <xah@xahlee.org>" -1000 nil s)
  ("Xah <xahlee@gmail.com>" -1000 nil s)))

You're one rampant bloody morpher. Stop wriggling, lest I 
regexp you.

Phil
-- 
I tried the Vista speech recognition by running the tutorial. I was 
amazed, it was awesome, recognised every word I said. Then I said the 
wrong word ... and it typed the right one. It was actually just 
detecting a sound and printing the expected word! -- pbhj on /.

Re: emacs in text mode terminal paste problem (SecureCRT / xterm )

[emacsuser]

Peter Dyballa <Peter_Dyballa@Web.DE> writes:

> Am 23.11.2008 um 00:14 schrieb emacsuser@invalid.org:
>
>> An apostrophe that get pasted appears to invoke "Regexp I-search
>> backward" somehow.
>> The search of course usually fails, because it can't find all of the
>> text that follows.
>
>
> The so-called apostrophes from MS Word could be single quote
> characters from the area UTF+201x. Maybe it works to C-q in some
> (*scratch*) buffer and then paste just that "apostrophe." You might
> see the bytes that would be actually inserted. The three bytes of
> UTF-8 representation (E2 80 99 for ., RIGHT SINGLE QUOTATION MARK)
> can be interpreted as this: E2 is in 7-bit idiom C-r (single 8th bit
> set + remainder r), the other two bytes can be any 7-bit rubbish.
>
> GNU Emacs offers some "translations" for X selections . lookup
> info!  X11 also has some configuration options. A better option is to
> upgrade to GNU Emacs 22.3.

Thanks for the tips Peter (and all!).   Peter, I think you've got it.

I was just able to figure out where isearch-regexp-backards is
normally bound.  This apostrophe is somehow getting sent as
Ctrl-Meta-r  

C-q followed by pasting that apostrophe from Word results in a proper,
simple pasting of a '

But pasting just the apostrophe _without_ the C-q before it brings up:
        Regexp I-search backward:

And after some digging, I finally found where in the help that binding
is mentioned.  Apparently, it's Ctrl-Meta-r. 

Yes, I imagine and emacs upgrade is in order.  Pesky backports for
debian.   I'm not using X though in this situation -- SecureCRT into a
command prompt, and emacs -nw within that ssh terminal.  

Thanks again for the character set foo I was seeking to point me in
the direction of a Ctrl-r derivative.  

I'll followup if/when I tame what's going on but I've at least
isolated what it's getting translated to!