[Patch] Unicode support for the MS Windows clipboard

[Patch] Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

[-- Attachment #1: Type: text/plain, Size: 2412 bytes --]

Hi everybody,


As we just had another discussion on this, I sat down and wrote a
preliminary implementation of Unicode support for the MS Windows
clipboard, see attached patch.

I assume some of you have opinions on how this should be actually be
packaged, so this patch is mostly for introduction, testing and
playing around.  If this code is acceptable in principle, I will
probably also need to send in papers for the copyright assignment,
before it can be used.


What does it do:

- Introduce a new variable `w32-clipboard-type' to use with
  cut-and-paste instead of the hard-coded CF_TEXT.  The default for
  `w32-clipboard-type' is CF_TEXT, because CF_UNICODETEXT is not
  compatible with 9x/Me, it uses more memory and CF_TEXT was used
  before.

- Drop optimizations for ASCII-only text.  This is mostly because I
  couldn't get all combinations straight in my mind, between this, the
  `last_clipboard_text' mechanism and CF_UNICODETEXT.


Open questions:

- Support for CF_OEMTEXT (console text) in the clipboard may look
  superfluous.  OTOH this may be usefull for full console mode support
  (emacs -nw) on 9x/Me.  If we keep this, the default for
  `w32-clipboard-type' should depend on the display mode (console vs
  GUI).  In that case `w32-clipboard-type' should be set from the Lisp
  code, though, I think.

- `selection-coding-system' and `w32-clipboard-type' need to be set in
  a synchronized manner.  I'm not yet sure if we want this to be done
  by Lisp code or by combining/synchronizing the two variables in the
  C code somehow.

  If we keep them separate, users can in theory use other
  coding-systems with this as they see fit.  They can adapt it to
  whatever exotic locales and coding-systems that they may have.  But
  that may be an academic issue.

  Anyway, as long as they are kept separate, these are the recommended
  combinations for an English version of Windows:

      w32-clipboard-type | selection-coding-system
      --------------------------------------------
      CF_TEXT            | cp1252-dos (GetACP())
      CF_OEMTEXT         | cp850-dos (GetOEMCP())
      CF_UNICODETEXT     | utf-16le-dos

- If we keep `w32-clipboard-type', we could try to use more
  user-oriented names for the different types instead of just taking
  them from the C API names.

- We want to either drop the #if 0 sections in the code completly or
  make that code work.


benny


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: w32select.c.patch --]
[-- Type: text/x-patch, Size: 6725 bytes --]

Index: w32select.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/w32select.c,v
retrieving revision 1.32
diff -u -p -r1.32 w32select.c
--- w32select.c	18 Apr 2004 18:34:03 -0000	1.32
+++ w32select.c	26 May 2004 17:35:14 -0000
@@ -41,6 +41,10 @@ static Lisp_Object Vselection_coding_sys
 /* Coding system for the next communicating with other Windows programs.  */
 static Lisp_Object Vnext_selection_coding_system;
 
+/* Type of clipboard transfer method that should be used.  */
+static Lisp_Object Vw32_clipboard_type;
+static Lisp_Object QCF_TEXT, QCF_OEMTEXT, QCF_UNICODETEXT;
+
 /* Sequence number, used where possible to detect when we are pasting
    our own text.  */
 static DWORD last_clipboard_sequence_number;
@@ -110,6 +114,20 @@ DEFUN ("w32-close-clipboard", Fw32_close
 
 #endif
 
+static UINT
+get_cf_type (void)
+{
+  CHECK_SYMBOL (Vw32_clipboard_type);
+
+  if (EQ (Vw32_clipboard_type, QCF_UNICODETEXT))
+    return CF_UNICODETEXT;
+
+  if (EQ (Vw32_clipboard_type, QCF_OEMTEXT))
+    return CF_OEMTEXT;
+
+  return CF_TEXT;
+}
+
 DEFUN ("w32-set-clipboard-data", Fw32_set_clipboard_data,
        Sw32_set_clipboard_data, 1, 2, 0,
        doc: /* This sets the clipboard data to the given text.  */)
@@ -134,6 +152,7 @@ DEFUN ("w32-set-clipboard-data", Fw32_se
   src = SDATA (string);
   dst = src;
 
+#if 0 /* Disable ASCII-only optimizations */
   /* We need to know how many lines there are, since we need CRLF line
      termination for compatibility with other Windows Programs.
      avoid using strchr because it recomputes the length every time */
@@ -142,8 +161,10 @@ DEFUN ("w32-set-clipboard-data", Fw32_se
       nlines++;
       dst++;
     }
+#endif
 
   {
+#if 0 /* Disable ASCII-only optimizations */
     /* Since we are now handling multilingual text, we must consider
        encoding text for the clipboard.  */
     int charset_info = find_charset_in_text (src, SCHARS (string),
@@ -192,6 +213,7 @@ DEFUN ("w32-set-clipboard-data", Fw32_se
 	Vlast_coding_system_used = Qraw_text;
       }
     else
+#endif
       {
 	/* We must encode contents of OBJ to the selection coding
            system. */
@@ -242,7 +264,14 @@ DEFUN ("w32-set-clipboard-data", Fw32_se
 							 clipboard_storage_size);
 	      }
 	    if (last_clipboard_text)
-	      memcpy (last_clipboard_text, dst, coding.produced);
+	      {
+		memcpy (last_clipboard_text, dst, coding.produced);
+		/* Add a string terminator. Set *two* bytes after the
+		   string to NUL, the second just in case we are using
+		   CF_UNICODETEXT. */
+		last_clipboard_text[coding.produced] = 
+		  last_clipboard_text[coding.produced+1] = '\0';
+	      }
 	  }
 
 	GlobalUnlock (htext);
@@ -257,7 +286,7 @@ DEFUN ("w32-set-clipboard-data", Fw32_se
   if (!OpenClipboard ((!NILP (frame) && FRAME_W32_P (XFRAME (frame))) ? FRAME_W32_WINDOW (XFRAME (frame)) : NULL))
     goto error;
 
-  ok = EmptyClipboard () && SetClipboardData (CF_TEXT, htext);
+  ok = EmptyClipboard () && SetClipboardData (get_cf_type(), htext);
 
   CloseClipboard ();
 
@@ -277,8 +306,11 @@ DEFUN ("w32-set-clipboard-data", Fw32_se
 
   ok = FALSE;
   if (htext) GlobalFree (htext);
+
+  /* Set the first *two* bytes to NUL, the second just in case we are
+     using CF_UNICODETEXT. */
   if (last_clipboard_text)
-    *last_clipboard_text = '\0';
+    last_clipboard_text[0] = last_clipboard_text[1] = '\0';
 
   last_clipboard_sequence_number = 0;
 
@@ -296,6 +328,7 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
 {
   HANDLE htext;
   Lisp_Object ret = Qnil;
+  UINT cf_type;
 
   if (!NILP (frame))
     CHECK_LIVE_FRAME (frame);
@@ -305,7 +338,8 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
   if (!OpenClipboard ((!NILP (frame) && FRAME_W32_P (XFRAME (frame))) ? FRAME_W32_WINDOW (XFRAME (frame)) : NULL))
     goto done;
 
-  if ((htext = GetClipboardData (CF_TEXT)) == NULL)
+  cf_type = get_cf_type();
+  if ((htext = GetClipboardData (cf_type)) == NULL)
     goto closeclip;
 
   {
@@ -313,12 +347,17 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
     unsigned char *dst;
     int nbytes;
     int truelen;
+#if 0 /* Disable ASCII-only optimizations */
     int require_decoding = 0;
+#endif
 
     if ((src = (unsigned char *) GlobalLock (htext)) == NULL)
       goto closeclip;
 
-    nbytes = strlen (src);
+    if (cf_type == CF_UNICODETEXT)
+      nbytes = (lstrlenW ((WCHAR *)src) + 1) * 2;
+    else
+      nbytes = strlen (src) + 1;
 
     /* If the text in clipboard is identical to what we put there
        last time w32_set_clipboard_data was called, pretend there's no
@@ -332,6 +371,12 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
 	    && memcmp(last_clipboard_text, src, nbytes) == 0))
       goto closeclip;
 
+    /* Drop the string terminator from here on. */
+    nbytes --;
+    if (cf_type == CF_UNICODETEXT)
+      nbytes --;
+
+#if 0 /* Disable ASCII-only optimizations */
     {
       /* If the clipboard data contains any non-ascii code, we
 	 need to decode it.  */
@@ -346,8 +391,11 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
 	    }
 	}
     }
+#endif
 
+#if 0 /* Disable ASCII-only optimizations */
     if (require_decoding)
+#endif
       {
 	int bufsize;
 	unsigned char *buf;
@@ -376,6 +424,7 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
 	    && !NILP (Ffboundp (coding.post_read_conversion)))
 	  ret = run_pre_post_conversion_on_str (ret, &coding, 0);
       }
+#if 0 /* Disable ASCII-only optimizations */
     else
       {
 	/* Need to know final size after CR chars are removed because we
@@ -421,6 +470,7 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
 
 	Vlast_coding_system_used = Qraw_text;
       }
+#endif
 
     GlobalUnlock (htext);
   }
@@ -499,7 +549,21 @@ next communication only.  After the comm
 set to nil.  */);
   Vnext_selection_coding_system = Qnil;
 
-  QCLIPBOARD = intern ("CLIPBOARD");	staticpro (&QCLIPBOARD);
+  DEFVAR_LISP ("w32-clipboard-type", &Vw32_clipboard_type,
+	       doc: /* MS Windows clipboard type for the communication with other programs.
+When sending or receiving text via clipboard, this is the Windows text
+type that is used.  It can be set to `CF_TEXT', `CF_OEMTEXT' or
+`CF_UNICODETEXT'.  `CF_UNICODETEXT' is only valid on NT, Windows 2000
+or Windows XP.  `selection-coding-system' must be set to match.  The
+default value is `CF_TEXT'.  */);
+
+  QCF_TEXT	  = intern ("CF_TEXT");		staticpro (&QCF_TEXT);
+  QCF_OEMTEXT	  = intern ("CF_OEMTEXT");	staticpro (&QCF_OEMTEXT);
+  QCF_UNICODETEXT = intern ("CF_UNICODETEXT");	staticpro (&QCF_UNICODETEXT);
+  Vw32_clipboard_type = QCF_TEXT;
+
+
+  QCLIPBOARD	  = intern ("CLIPBOARD");	staticpro (&QCLIPBOARD);
 }
 
 /* arch-tag: c96e9724-5eb1-4dad-be07-289f092fd2af

[-- Attachment #3: Type: text/plain, Size: 141 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: [Patch] Unicode support for the MS Windows clipboard

[Jason Rumney]

Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:

> As we just had another discussion on this, I sat down and wrote a
> preliminary implementation of Unicode support for the MS Windows
> clipboard, see attached patch.

Thank you, this was the next thing on my todo list. The recent
changes to use cpXXXX as default coding-system on Windows were
intended to make this easier (see my CF_LOCALE comment below).

Your patch doesn't seem to contain any special handling for the
different types, so it seems that the user must take care to set
selection-coding-system appropriately for w32-clipboard-type. If that
is the case, it might be better to get rid of w32-clipboard-type as a
user variable, and determine the type automatically from
selection-coding-system instead. cp<900 should map to OEM, utf16
to unicode, and others to ANSI. Maybe it would be useful to map utf8
to utf16, and the various utf-16 endians to the correct one (I can
never remember which Windows uses, I would be surprised if most users
knew).

Also, we should set (and read) CF_LOCALE when we are using CF_TEXT, to
indicate the coding we have used. This will be easy when clipboard
coding system is cpXXXX, so even if we just do it in those cases
would be an improvement.


Here are some thoughts I'd had:

When reading from the clipboard, if CF_UNICODE is present, it might be
better to use that (ignoring selection-coding-system). On the other
hand, some Chinese characters are still not covered by Emacs' unicode
support (even with utf-translate-cjk-mode), so it might be better to
use CF_TEXT when selection-coding-system is set to one of the CJK
coding-systems (I am not sure about Japanese and Simplified Chinese
coverage, Korean is probably OK if they are only using Hangeul but
they might use the odd Hanja, Big5 is definitely not entirely
covered).

When writing to the clipboard, if we know CF_UNICODE is supported,
using it might produce better results, but if text can be encoded in
selection-coding-system, then using that and setting CF_LOCALE
appropriately will be guaranteed to work on more platforms.

Re: [Patch] Unicode support for the MS Windows clipboard

[Eli Zaretskii]

> From: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de>
> Date: Wed, 26 May 2004 20:01:22 +0200
> 
> As we just had another discussion on this, I sat down and wrote a
> preliminary implementation of Unicode support for the MS Windows
> clipboard, see attached patch.

Thanks!

> - Introduce a new variable `w32-clipboard-type' to use with
>   cut-and-paste instead of the hard-coded CF_TEXT.  The default for
>   `w32-clipboard-type' is CF_TEXT, because CF_UNICODETEXT is not
>   compatible with 9x/Me, it uses more memory and CF_TEXT was used
>   before.

Couldn't this be done without introducing Windows-specific options?
AFAIK, the logic employed by Windows when it encodes clipboard text is
quite simple, something like: if it cannot be encoded with the system
codepage, use Unicode.  Why cannot Emacs simply follow this logic?

Also, AFAIK CF_UNICODETEXT _can_ be used on Windows 9x, as any
program like clipbrd.exe or ClipConvert will show you.

> - Drop optimizations for ASCII-only text.  This is mostly because I
>   couldn't get all combinations straight in my mind, between this, the
>   `last_clipboard_text' mechanism and CF_UNICODETEXT.

Is that optimization indeed an optimization?  That is, does it give us
any significant speed-up?  Can you measure the speed of pasting a
large chunk of ASCII text in an Emacs that has this optimization and
compare it to the speed of pasting non-ASCII text?

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

Hi Eli,


>> From: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de>
>> - Introduce a new variable `w32-clipboard-type'

"Eli Zaretskii" <eliz@gnu.org> writes:
> Couldn't this be done without introducing Windows-specific options?

I'd certainly love to.  In general all three modes have their use:

- CF_TEXT - This is obviously the fallback. 

- CF_UNICODETEXT - The most capable choice on NT/W2K/XP.  According to
  MSDN not supported on 9x/Me.

- CF_OEMTEXT - If you want to cut-and-paste line drawing characters
  between Emacs and other console apps on 9x/Me this would be the type
  to use.

  You could consider this scenario too exotic, so that we could drop
  it.  OTOH I know of at least one user that is actually using and
  maintaining files with line drawing characters in Emacs.

> AFAIK, the logic employed by Windows when it encodes clipboard text
> is quite simple, something like: if it cannot be encoded with the
> system codepage, use Unicode.  Why cannot Emacs simply follow this
> logic?

Conceptually, Windows doesn't encode, it just marshals memory blocks.

Windows *will* automatically generate any additional supported text
type from whatever an application provides, so that an application
only needs to know one of the three text types.  In this context at
least, 9x/Me only supports CF_TEXT and CF_OEMTEXT while Windows
NT/W2K/XP also supports CF_UNICODETEXT (or so MSDN says).

I don't have a 9x machine ready here at the moment, but I will
probably install one on the weekend for testing of other things
anyway.

> Also, AFAIK CF_UNICODETEXT _can_ be used on Windows 9x, as any
> program like clipbrd.exe or ClipConvert will show you.

9x/Me is explicitly documented on MSDN not to support that.  Being
that - from Windows' POV - we are just talking about memory blocks,
CF_UNICODETEXT is probably marshalled fine, but is it also
automatically converted?  I.e. will any non-Unicode application be
able to retrieve the CF_TEXT format that it is entitled to expect,
when we just post CF_UNICODETEXT?  And the other way around?

Even if not we could probably try a scheme similar to what you
outlined above:

- Receiving:  First try CF_UNICODETEXT.  If CF_UNICODETEXT doesn't
  exist, try CF_TEXT. 

- Posting: Post CF_UNICODETEXT.  Test if CF_TEXT is there now.  If
  CF_TEXT is not automatically provided by Windows, post CF_TEXT
  ourself in additiona to CF_UNICODETEXT.  Note that this last
  situation would triple the amount of memory required.


Anyway, what happens to the MULE problem in this unified scenario?  Do
all problems go away with unify-8859-on-{de,en}coding?


>> - Drop optimizations for ASCII-only text.

> Is that optimization indeed an optimization?

It was obviously intented as such, but it may just have been a remnant
of the code that was there before the introduction of the
{en,de}coding via coding systems into that module.  I will build a
version without my patch and test it.


Thanks for your input,
benny

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

Hi Jason,


Jason Rumney <jasonr@gnu.org> writes:
> it seems that the user must take care to set selection-coding-system
> appropriately for w32-clipboard-type.

Right.  As I said the code is for experimentation right now.

> If that is the case, it might be better to get rid of
> w32-clipboard-type as a user variable, and determine the type
> automatically from selection-coding-system instead. cp<900 should
> map to OEM, utf16 to unicode, and others to ANSI.

Can we just assume this?  Does "cp<900" really garantee OEM?  How do
we know that some exotic private trick coding system isn't usefull for
CF_UNICODETEXT, or for CF_OEMTEXT, or do we just ignore that
possibility?  (Which is fine by me, don't get me wrong, users have the
source after all.)

> Also, we should set (and read) CF_LOCALE when we are using CF_TEXT,
> to indicate the coding we have used.

I'll have to look that up, I'm not familiar with CF_LOCALE.

> When reading from the clipboard, if CF_UNICODE is present, it might
> be better to use that (ignoring selection-coding-system).

Could we get into trouble with the MULE problem here?  Or does
unify-8859-on-{en,de}coding solve this for all cases?

> On the other hand, some Chinese characters are still not covered by
> Emacs' unicode support (even with utf-translate-cjk-mode), [...]
> Big5 is definitely not entirely covered).

If those characters are not supported by Unicode, how does Windows
support them, which is based on Unicode after all?  Does it support
them at all?  Or does it use the private characters for this?


Thanks,
benny


PS: The default for selection-coding-system should be cpXXXX-dos, not
just cpXXXX.  Otherwise I get <LF> as line ends instead of <CR><LF>
when I copy non-ASCII text.  Which than doesn't work well with
Notepad, of course.

Re: Unicode support for the MS Windows clipboard

[Jason Rumney]

Benjamin Riefenstahl wrote:

> Jason Rumney <jasonr@gnu.org> writes:

>>If that is the case, it might be better to get rid of
>>w32-clipboard-type as a user variable, and determine the type
>>automatically from selection-coding-system instead. cp<900 should
>>map to OEM, utf16 to unicode, and others to ANSI.
> 
> Can we just assume this?  Does "cp<900" really garantee OEM?

Possibly Thai is an exception, and maybe Vietnamese, we can make 
exceptions where necessary, but basically all the OEM codepages that are 
not also used as ANSI codepages are in the sub 900 range. The DBCS 
codepages in the 900-1000 range are used for both ANSI and OEM 
codepages, so either CF_TEXT or CF_OEMTEXT would be valid for them, 
though CF_TEXT is probably more widely recognized.

> How do we know that some exotic private trick coding system isn't usefull for
> CF_UNICODETEXT

The encoding of CF_UNICODETEXT does not vary, so utf-16-le (or maybe 
-be) is the only coding-system that is appropriate. As mentioned, we 
could map other utf coding systems automatically onto the right one, to 
avoid the user having to know too many details.

 > or for CF_OEMTEXT

CF_OEMTEXT is defined as the default console codepage for that version 
of Windows. Although it is theoretically possible for the user to have 
customized it beyond the limited set that come out of the box with 
different localised versions of Windows, it really isn't that 
interesting to us because other applications probably wouldn't support 
those non-default encodings either. I doubt there are many (if any) 
applications that support CF_OEMTEXT but not CF_TEXT, so it is probably 
better to just ignore it until someone comes up with a reason why we 
should support it.

>>Also, we should set (and read) CF_LOCALE when we are using CF_TEXT,
>>to indicate the coding we have used.
> 
> I'll have to look that up, I'm not familiar with CF_LOCALE.

I think the problem I had with that was finding a locale given an ANSI 
codepage. In the case where CF_LOCALE is the default system locale, we 
don't need to set it, and in other cases we would be better using 
CF_UNICODETEXT, so maybe this is not worth pursuing.

>>When reading from the clipboard, if CF_UNICODE is present, it might
>>be better to use that (ignoring selection-coding-system).
> 
> Could we get into trouble with the MULE problem here?  Or does
> unify-8859-on-{en,de}coding solve this for all cases?
> 
>>On the other hand, some Chinese characters are still not covered by
>>Emacs' unicode support (even with utf-translate-cjk-mode), [...]
>>Big5 is definitely not entirely covered).
> 
> If those characters are not supported by Unicode, how does Windows
> support them, which is based on Unicode after all?  Does it support
> them at all?  Or does it use the private characters for this?

Maybe I am imagining a problem that is not there. Having checked again, 
it seems the problem I saw with a character not being displayed, which I 
thought was due to an unsupported character was actually due to a 
character (in Chinese Traditional text) being decoded as 
japanese-jisx0212 which I don't have a font for. I can still see this 
being a major problem for Chinese users though.


   character: [] (0254137, 88159, 0x1585f, U+4F60)
     charset: japanese-jisx0212 (JISX0212 Japanese supplement: ISO-IR-159.)
  code point: 48 95
      syntax: w 	which means: word
    category: C:Chinese (Han) characters of 2-byte character sets 
j:Japanese
              |:While filling, we can break a line at this character.
buffer code: 0x94 0xB0 0xDF
   file code: not encodable by coding system mule-utf-8-dos
     display: no font available


> PS: The default for selection-coding-system should be cpXXXX-dos, not
> just cpXXXX.  Otherwise I get <LF> as line ends instead of <CR><LF>
> when I copy non-ASCII text.  Which than doesn't work well with
> Notepad, of course.

Thanks, the code to make sure selection-coding-system was dos seems to 
have been removed in my previous changes.

Re: Unicode support for the MS Windows clipboard

[Stefan Monnier]

> Maybe I am imagining a problem that is not there. Having checked again, it
> seems the problem I saw with a character not being displayed, which
> I thought was due to an unsupported character was actually due to
> a character (in Chinese Traditional text) being decoded as japanese-jisx0212
> which I don't have a font for. I can still see this being a major problem
> for Chinese users though.

Isn't that because the default is to assume a japanese environment?
Have you tried to set current-language-environment to something
like "Chinese-BIG5" or "Chinese-GB" ?

Also how does the CF_TEXT currently work for those chinese users?


        Stefan

Re: [Patch] Unicode support for the MS Windows clipboard

[Stefan Monnier]

> Couldn't this be done without introducing Windows-specific options?
> AFAIK, the logic employed by Windows when it encodes clipboard text is
> quite simple, something like: if it cannot be encoded with the system
> codepage, use Unicode.  Why cannot Emacs simply follow this logic?

Agreed.  This way the user does not need to change anything
(selection-coding-system works just as before) and that things that used to
work still work.

>> - Drop optimizations for ASCII-only text.  This is mostly because I
>> couldn't get all combinations straight in my mind, between this, the
>> `last_clipboard_text' mechanism and CF_UNICODETEXT.
> Is that optimization indeed an optimization?

I strongly suspect that any potential difference in speed is lost in
the noise.


        Stefan

Re: Unicode support for the MS Windows clipboard

[Jason Rumney]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Isn't that because the default is to assume a japanese environment?

My locale was set to Japanese, so it might be appropriate to prefer
JISX0212 over Big5 (I'm not sure though - how widespread is JISX0212
support even within Japan?  Windows only seems to have support for
JISX0208.1990 AFAICT).

I am not sure what it does in a European locale, but Big5 would be
more appropriate there IMHO.

> Have you tried to set current-language-environment to something
> like "Chinese-BIG5" or "Chinese-GB" ?
>
> Also how does the CF_TEXT currently work for those chinese users?

Since recently, Emacs uses Big5 (cp950) or GB (cp936) out of the
box. Previously the user could set selection-coding-system themselves
to get the right thing, but the default in all locales was latin-1,
which was only really appropriate for users from English speaking
countries (excluding Eire).

Re: Unicode support for the MS Windows clipboard

[Jason Rumney]

Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:

> - Posting: Post CF_UNICODETEXT.  Test if CF_TEXT is there now.  If
>   CF_TEXT is not automatically provided by Windows, post CF_TEXT
>   ourself in additiona to CF_UNICODETEXT.  Note that this last
>   situation would triple the amount of memory required.

Andrew Innes always had the intention to make the clipboard work
on-demand, the same way it does on X. So the memory would only be
used if the clipboard text was actually pasted (and then only for the
format the client wanted).

Another thing worth considering, if we are making major changes to the
clipboard code, is that Kenichi Handa pointed out some time ago that
the encoding part of the X clipboard support is now done in Lisp
(xselect.el). Windows could do this too.

But now we are in feature freeze. It is probably worth letting Emacs
support CF_UNICODETEXT in some way, but I think the ideal solution is
too much. If you can find a simple solution that does not introduce
new user variables, even if it requires manual work by the user to
force unicode, then lets install that.

> Anyway, what happens to the MULE problem in this unified scenario?  Do
> all problems go away with unify-8859-on-{de,en}coding?

What MULE problem?

Re: Unicode support for the MS Windows clipboard

[Eli Zaretskii]

> From: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de>
> Date: Thu, 27 May 2004 11:45:42 +0200
> 
> - CF_OEMTEXT - If you want to cut-and-paste line drawing characters
>   between Emacs and other console apps on 9x/Me this would be the type
>   to use.

Doesn't Unicode support line-drawing characters?

> CF_UNICODETEXT is probably marshalled fine, but is it also
> automatically converted?  I.e. will any non-Unicode application be
> able to retrieve the CF_TEXT format that it is entitled to expect,
> when we just post CF_UNICODETEXT?  And the other way around?

Sorry, I don't know the answers to these questions.  We will have to
test that, or ask experts, or google for the info.

> Even if not we could probably try a scheme similar to what you
> outlined above:
> 
> - Receiving:  First try CF_UNICODETEXT.  If CF_UNICODETEXT doesn't
>   exist, try CF_TEXT. 
> 
> - Posting: Post CF_UNICODETEXT.  Test if CF_TEXT is there now.  If
>   CF_TEXT is not automatically provided by Windows, post CF_TEXT
>   ourself in additiona to CF_UNICODETEXT.  Note that this last
>   situation would triple the amount of memory required.

I'd say that for posting, we should try to use selection-coding-system
first, if it makes sense (for some definition of ``make sense'' that
we will have to come up with).  Especially, if selection-coding-system
is the same as the system codepage, we should use CF_TEXT, as that
will, AFAIK, allow more (older) applications to be able to paste.

For receiving, I think the logic you suggest is okay, but we probably
should leave some provision to force Emacs to use CF_TEXT/CF_OEMTEXT
and a specific encoding, for borderline cases that we will never be
able to forsee.

> Anyway, what happens to the MULE problem in this unified scenario?  Do
> all problems go away with unify-8859-on-{de,en}coding?

They should, I think, but only time and experience will tell.  That's
why an option to force Emacs to use CF_TEXT on receive might come in
handy.

> >> - Drop optimizations for ASCII-only text.
> 
> > Is that optimization indeed an optimization?
> 
> It was obviously intented as such, but it may just have been a remnant
> of the code that was there before the introduction of the
> {en,de}coding via coding systems into that module.

I think it's there because the original code written for X had it, and
the Windows port simply copied it (as did the DOS port, see
w16select.c).

Decoding obviously slows down things, but, as Stefan pointed out, the
slow-down could well be below the noise level.

> I will build a version without my patch and test it.

Thanks.

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

Hi all,


I have been doing some testing, research and thinking.  First here is
a quick response on a few points.  Next I'll do a new patch based on
my current ideas.


> "Eli Zaretskii" <eliz@gnu.org> writes:
>> Couldn't this be done without introducing Windows-specific options?

Jason Rumney <jasonr@gnu.org> writes:
> Also, we should set (and read) CF_LOCALE when we are using CF_TEXT,
> to indicate the coding we have used.

Thanks for the pointer, researching locales actually led me to a
solution for deriving codepage properties (OEM vs "ANSI") via locales.

I think I have an algorithm that works.  It makes a few assumptions
about the coding system names and based on that it derives the
requested clipboard type automatically.

How does this sound:

- If selection-coding-system has the form /(.*-)?utf-16.*/, I assume
  CF_UNICODETEXT is wanted.

- If selection-coding-system has the form /cp[0-9]+.*/ or
  /windows-[0-9]+.*/, I derive the codepage from that.

    - Check if the codepage is identical to GetACP() or GetOEMCP().
      If it is, use CF_TEXT or CF_OEMTEXT accordingly. 

    - Else get a corresponding LCID (reverse mapping via
      EnumLocales()) which has the codepage as OEM or "ANSI".  In this
      case we also need to set LC_LOCALE accordingly.

The last step takes a small performance hit, but the results can
easily be cached.

I am also thinking of custom coding systems, like e.g. for doing
automatic remapping of private characters or locale specific
pre-/postprocessing.  This is why I am not completely comfortable with
hardcoding coding systems or using heuristics based on the coding
system symbol names.  If such concerns are completely misplaced,
please just tell ;-).

Anyway, I have no problem with dictating the above naming conventions
for selection-coding-system for now.


Jason Rumney <jasonr@gnu.org> writes:
> Andrew Innes always had the intention to make the clipboard work
> on-demand, the same way it does on X. So the memory would only be
> used if the clipboard text was actually pasted (and then only for
> the format the client wanted).

We could do that using WM_RENDERFORMAT.  But than we absolutely need a
valid HWND to get a target for that message.  I don't know anything
about the Emacs message loop and the windows that are available.  It
would probably be best to allocate a custom hidden window for this.
I'll postpone that idea for now and just assume that we don't use
Unicode on 9x/Me.

Jason Rumney <jasonr@gnu.org> writes:
> Another thing worth considering, if we are making major changes to the
> clipboard code, is that Kenichi Handa pointed out some time ago that
> the encoding part of the X clipboard support is now done in Lisp
> (xselect.el). Windows could do this too.

At the moment this is done via {de,en}code_coding() and a couple of
friends.  Is that the same thing?


Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:
>> Anyway, what happens to the MULE problem in this unified scenario?
>> Do all problems go away with unify-8859-on-{de,en}coding?

Jason Rumney <jasonr@gnu.org> writes:
> What MULE problem?

Disjunct charsets leading to the introduction of unwanted characters
(similar to that SHIFT-JIS <-> Chinese confusion that you just
mentioned).  At one of the last times when the discussion came up
somebody mentioned that this could still be a serious problem.


Jason Rumney <jasonr@gnu.org> writes:
> The encoding of CF_UNICODETEXT does not vary, so utf-16-le (or maybe
> -be) is the only coding-system that is appropriate.

Actually at the moment that would be utf-16le-dos, not utf-16-le-dos.
The latter includes a BOM, which we really don't want here.  The
non-intuitive naming difference makes me wonder though, if this is
just some unintended confusion?  There are also currently
utf-16-le-with-signature-* and mule-utf-16-*.


> "Eli Zaretskii" <eliz@gnu.org> writes:
>> Also, AFAIK CF_UNICODETEXT _can_ be used on Windows 9x, as any
>> program like clipbrd.exe or ClipConvert will show you.

I tested Win95 and Win98SE.  On both systems, the clipboard viewer and
Notepad couldn't make use of CF_UNICODETEXT.  Cut-and-paste between
two Emacs instances via CF_UNICODETEXT works, so i assume other
applications that support CF_UNICODETEXT would work, too.  No
automatic conversion by Windows, though.


Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:
>>> - Drop optimizations for ASCII-only text.

> "Eli Zaretskii" <eliz@gnu.org> writes:
>> Is that optimization indeed an optimization?

Getting data from the clipboard is indeed quite a bit faster with this
optimization.  Putting something on the clipboard doesn't benefit, but
that's probably because the detection of this case is inefficient, it
uses find_charset_in_text(), although the result is not really
used. So probably that can be made better, too.  I'll try to get this
integrated in the next version of the patch.


benny

Re: Unicode support for the MS Windows clipboard

[Jason Rumney]

Benjamin Riefenstahl wrote:

>>Another thing worth considering, if we are making major changes to the
>>clipboard code, is that Kenichi Handa pointed out some time ago that
>>the encoding part of the X clipboard support is now done in Lisp
>>(xselect.el). Windows could do this too.

Sorry, it is select.el, not xselect.el, though it looks pretty X 
specific, so I don't think it could be reused directly.

> At the moment this is done via {de,en}code_coding() and a couple of
> friends.  Is that the same thing?

No, that is something different, but lets put this bit off until the 
on-demand clipboard changes, which are going to be major anyway. This 
might all become much simpler once the unicode branch is merged anyway.

> Jason Rumney <jasonr@gnu.org> writes:
> 
>>The encoding of CF_UNICODETEXT does not vary, so utf-16-le (or maybe
>>-be) is the only coding-system that is appropriate.
> 
> 
> Actually at the moment that would be utf-16le-dos, not utf-16-le-dos.

That's why I suggested making all utf-16 coding systems (maybe even 
utf-8) mean "use CF_UNICODETEXT". Even I do not understand all the 
subtle differences between them and which is appropriate to use for 
Windows clipboard, so we can't expect average users to.

Re: Unicode support for the MS Windows clipboard

[Stefan Monnier]

> - CF_OEMTEXT - If you want to cut-and-paste line drawing characters
>   between Emacs and other console apps on 9x/Me this would be the type
>   to use.

>   You could consider this scenario too exotic, so that we could drop
>   it.  OTOH I know of at least one user that is actually using and
>   maintaining files with line drawing characters in Emacs.

I don't think we should worry about a few users using line drawing
characters in 9x/Me for whom copy/paste didn't work before either.
I think we should concentrate on:
1 - make sure things that used to work still work.
2 - get new cases working by using CF_UNICODETEXT.

That basically means that we should always first try to use CF_TEXT like we
used to, and if that doesn't work try CF_UNICODETEXT.

Now for posting, it's OK because we can easily figure out whether CF_TEXT
can be used (just see whether encoding for CF_TEXT finds any un-encodable
chars).  For receiving, I don't know: can we ask Windows to tell us whether
the CF_TEXT we find contains "un-encoded chars" in which case we should look
at CF_UNICODETEXT?  If not we'd have to first try CF_UNICODETEXT with the
associated problems linked to unification this entails.


        Stefan

Re: Unicode support for the MS Windows clipboard

[Kenichi Handa]

In article <40B75132.4000207@gnu.org>, Jason Rumney <jasonr@gnu.org> writes:

> Benjamin Riefenstahl wrote:
>>> Another thing worth considering, if we are making major changes to the
>>> clipboard code, is that Kenichi Handa pointed out some time ago that
>>> the encoding part of the X clipboard support is now done in Lisp
>>> (xselect.el). Windows could do this too.

> Sorry, it is select.el, not xselect.el, though it looks pretty X 
> specific, so I don't think it could be reused directly.

I'm not suggesting to resuse select.el, but to use the
similar mechanism for converting code in Lisp.

> No, that is something different, but lets put this bit off until the 
> on-demand clipboard changes, which are going to be major anyway. This 
> might all become much simpler once the unicode branch is merged anyway.

I agree.

---
Ken'ichi HANDA
handa@m17n.org

Re: Unicode support for the MS Windows clipboard

[Eli Zaretskii]

> From: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de>
> Date: Fri, 28 May 2004 15:26:10 +0200
> 
> How does this sound:
> 
> - If selection-coding-system has the form /(.*-)?utf-16.*/, I assume
>   CF_UNICODETEXT is wanted.
> 
> - If selection-coding-system has the form /cp[0-9]+.*/ or
>   /windows-[0-9]+.*/, I derive the codepage from that.
> 
>     - Check if the codepage is identical to GetACP() or GetOEMCP().
>       If it is, use CF_TEXT or CF_OEMTEXT accordingly. 
> 
>     - Else get a corresponding LCID (reverse mapping via
>       EnumLocales()) which has the codepage as OEM or "ANSI".  In this
>       case we also need to set LC_LOCALE accordingly.

That's okay, I think, if we also set the default value of
selection-coding-system on Windows to utf-16-le-dos (or what's its
name).

Re: Unicode support for the MS Windows clipboard

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: 28 May 2004 11:18:32 -0400
> 
> For receiving, I don't know: can we ask Windows to tell us whether
> the CF_TEXT we find contains "un-encoded chars" in which case we
> should look at CF_UNICODETEXT?

I don't think we can.  But we could look for "?" signs at unusual
places and decide they mean a failure of such kind, I guess.

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

Hi Eli,


>> From: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de>
>> - CF_OEMTEXT - [...] line drawing characters [...] 9x/Me

"Eli Zaretskii" <eliz@gnu.org> writes:
> Doesn't Unicode support line-drawing characters?

>> CF_UNICODETEXT [on 9x/Me ...]

As I said in the other post, applications on 95 and 98 do not support
CF_UNICODETEXT and the format is not automatically converted by the
system as it is on NT/W2K/XP.  This is as documented on MSDN.

> [...] if selection-coding-system is the same as the system codepage,
> we should use CF_TEXT, as that will, AFAIK, allow more (older)
> applications to be able to paste.

That is the case with the algorithm I outlined in my other mail.


benny

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

Hi Eli,

>> From: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de>
>> [deriving clipboard format from selection-coding-system]

"Eli Zaretskii" <eliz@gnu.org> writes:
> That's okay, I think, if we also set the default value of
> selection-coding-system on Windows to utf-16-le-dos (or what's its
> name).

I'd suggest doing that only on NT/W2K/XP.  On 9x/Me we should set it
based on the current code page as it is currently.  Setting Unicode by
default on 9x/Me would fail there when we post in that format and
other applications don't recognise CF_UNICODETEXT.

We can make it better when we implement delayed rendering.

benny

Re: Unicode support for the MS Windows clipboard

[Eli Zaretskii]

> From: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de>
> Date: Sat, 29 May 2004 16:52:52 +0200
> 
> "Eli Zaretskii" <eliz@gnu.org> writes:
> > That's okay, I think, if we also set the default value of
> > selection-coding-system on Windows to utf-16-le-dos (or what's its
> > name).
> 
> I'd suggest doing that only on NT/W2K/XP.  On 9x/Me we should set it
> based on the current code page as it is currently.

Agreed.

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

[-- Attachment #1: Type: text/plain, Size: 4161 bytes --]

Hi all,


Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:
> - If selection-coding-system has the form /(.*-)?utf-16.*/, I assume
>   CF_UNICODETEXT is wanted.
>
> - If selection-coding-system has the form /cp[0-9]+.*/ or
>   /windows-[0-9]+.*/, I derive the codepage from that.
>
>     - Check if the codepage is identical to GetACP() or GetOEMCP().
>       If it is, use CF_TEXT or CF_OEMTEXT accordingly. 
>
>     - Else get a corresponding LCID (reverse mapping via
>       EnumLocales()) which has the codepage as OEM or "ANSI".  In this
>       case we also need to set LC_LOCALE accordingly.

I implemented this in the attached patch.  I have tested it on W2K, 95
and 98SE.  Please tell of any problem the code has or of other
improvements I can make.

I have a couple of notes:


The C code constructs it's own cpXXXX-dos coding system names in one
place.  I would like to check that the constructed coding system
actually exists, but I don't know how yet.

I didn't worry about the console encoding (CF_OEMTEXT) too much, but I
added support for it, where it seemed straight-forward to me.

I get the feeling that the C code does too much, and that it should
delegate major parts of the processing to Lisp.  I am not yet sure how
to structure that, though.

Somebody suggested to automatically map all Unicode coding-systems to
the right one, utf-16le-dos.  If we actually want that, I'd suggest
doing it in the Lisp in (set-selection-coding-system) and not in C.  I
also wouldn't want to automatically map utf-8-dos, because it seems
clear and well-known to me that an 8-bit Unicode variant is not what
Windows thinks of as "Unicode", so when I say utf-8-dos, I expect that
to be respected here.  But that's just IMO.

I didn't do anything about the defaults yet.  That's because it's done
in Lisp until now, and I haven't studied that code yet.  To reiterate
the last discussion, the idea would be to set selection-coding-system
to utf-16le-dos on NT/W2K/XP and to cpXXXX on 9x/Me.

About the ASCII-only optimizations in w32select.c, these are now
enabled, except for the Unicode case in w32-get-clipboard-data, where
I would have to duplicate some 8-bit code as 16-bit code.  I didn't
feel like doing that just now.

Even with a small enhancement, the optimization still doesn't do very
much for setting the clipboard.  It *does* make a noticable difference
with *getting* data from the clipboard.  Ideally somebody could
profile w32-set-clipboard-data to see if the time is spent in Emacs or
in Windows' clipboard API.

Unicode on 9x: The only applications that I have on 98 that have
support for CF_UNICODETEXT are Wordpad and IE 6, they produce that
format themself and they also use it if somebody has posted it.  I
assume Word and other Office apps would also have it.  Both Wordpad
and IE need CF_TEXT to be present (or promised via delayed rendering)
to even enable pasting, so Unicode pasting has to wait until we
implement delayed rendering in Emacs.

We'll probably need documentation updates in some place. 

The patch also fixes two mostly benign bugs.


I assume that for this to go in, I need to sign papers, and I
understand that somebody needs to contact me about that from your
side.


While I am still familiar with the internal of this module, I'd like
to tackle delayed rendering.  I am thinking of constructing a hidden
window for hanging the message handlers on (we have a regular message
loop running even with "emacs -nw", right?).  I know how to do that,
so it should not be too much of an effort.


benny


2004-06-02  Benjamin Riefenstahl  <Benjamin.Riefenstahl@epost.de>

	* w32select.c (cached_coding_system, codepage, lcid)
	(clipboard_type, DEFAULT_LCID, ANSICP, OEMCP): New static
	variables.
	(cp_from_locale, enum_locale_callback, setup_parameters): New
	functions.
	(Fw32_set_clipboard_data): Handle CF_UNICODETEXT and CF_LOCALE.
	Correct end-of-string handling on clipboard.
	(Fw32_get_clipboard_data): Handle CF_UNICODETEXT and CF_LOCALE.
	Correct end-of-string for last_clipboard_text.
	(Fx_selection_exists_p): Handle CF_UNICODETEXT.
	(syms_of_w32select): Init and register cached_coding_system.
	

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: w32select.c.patch --]
[-- Type: text/x-patch, Size: 21380 bytes --]

Index: w32select.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/w32select.c,v
retrieving revision 1.32
diff -u -p -r1.32 w32select.c
--- w32select.c	18 Apr 2004 18:34:03 -0000	1.32
+++ w32select.c	2 Jun 2004 15:17:41 -0000
@@ -41,6 +41,16 @@ static Lisp_Object Vselection_coding_sys
 /* Coding system for the next communicating with other Windows programs.  */
 static Lisp_Object Vnext_selection_coding_system;
 
+/* Cached transfer parameters.  */
+static Lisp_Object cached_coding_system;
+static UINT codepage;
+static LCID lcid;
+static UINT clipboard_type;
+
+/* Constants, initialized dynamically later. */
+static LCID DEFAULT_LCID = LOCALE_NEUTRAL;
+static UINT ANSICP, OEMCP;
+
 /* Sequence number, used where possible to detect when we are pasting
    our own text.  */
 static DWORD last_clipboard_sequence_number;
@@ -51,7 +61,17 @@ extern ClipboardSequence_Proc clipboard_
    use data put on the clipboard by Emacs because the clipboard data
    could be MULEtilated by inappropriately chosen
    (next-)selection-coding-system.  For this reason, we must store the
-   text *after* it was encoded/Unix-to-DOS-converted.  */
+   text *after* it was encoded/Unix-to-DOS-converted.
+
+   FIXME: This doesn't work quite right, in the situation when the
+   last stored text is in CF_TEXT format and the format that we want
+   according to `selection-coding-system' is CF_UNICODETEXT.  We'd
+   have to retrieve the CF_TEXT just for the comparison, which is
+   wastefull with delayed rendering.  A better method is to check the
+   clipboard owner, but that requires that we *always* pass a HWND to
+   OpenClipboard(), and than preferably always the same HWND.  This
+   will probably be easier once we implement delayed rendering
+   (WM_RENDERFORMAT) ourself, where this is also a requirement.  */
 static unsigned char *last_clipboard_text = NULL;
 static size_t clipboard_storage_size = 0;
 
@@ -110,6 +130,136 @@ DEFUN ("w32-close-clipboard", Fw32_close
 
 #endif
 
+static UINT
+cp_from_locale (LCID lcid, UINT variant)
+{
+  char buffer[20] = "";
+  UINT cp;
+
+  GetLocaleInfo (lcid, variant, buffer, sizeof (buffer));
+  cp = strtoul (buffer, NULL, 10);
+
+  if (cp == CP_ACP)
+    return ANSICP;
+  else if (cp == CP_OEMCP)
+    return OEMCP;
+  else
+    return cp;
+}
+
+static BOOL WINAPI
+enum_locale_callback (/*const*/ char* loc_string)
+{
+  LCID this_lcid;
+  UINT this_codepage;
+
+  this_lcid = strtoul (loc_string, NULL, 16);
+
+  /* Is the wanted codepage the "ANSI" codepage for this locale? */
+  this_codepage = cp_from_locale (this_lcid, LOCALE_IDEFAULTANSICODEPAGE);
+  if (this_codepage == codepage)
+    {
+      lcid = this_lcid;
+      clipboard_type = CF_TEXT;
+      return FALSE; /* Stop enumeration */
+    }
+  
+  /* Is the wanted codepage the OEM codepage for this locale? */
+  this_codepage = cp_from_locale (this_lcid, LOCALE_IDEFAULTCODEPAGE);
+  if (this_codepage == codepage)
+    {
+      lcid = this_lcid;
+      clipboard_type = CF_OEMTEXT;
+      return FALSE; /* Stop enumeration */
+    }
+
+  return TRUE; /* Continue enumeration */
+}
+
+static void
+setup_parameters (void)
+{
+  const char *coding_name;
+  const char *cp;
+  char *end;
+  int slen;
+  Lisp_Object current_coding_system;
+
+  CHECK_SYMBOL (Vselection_coding_system);
+
+  /* One-time init. */
+  if (DEFAULT_LCID == LOCALE_NEUTRAL)
+    {
+      DEFAULT_LCID = GetUserDefaultLCID ();
+
+      /* Drop the sort order, so we compare this to CF_LOCALE objects
+	 with the same fix on W'9x.  */
+      DEFAULT_LCID = MAKELCID (LANGIDFROMLCID (DEFAULT_LCID), SORT_DEFAULT);
+
+      ANSICP = GetACP ();
+      OEMCP = GetOEMCP ();
+    }
+
+  /* Check if we have it cached */
+  current_coding_system = NILP (Vnext_selection_coding_system) ?
+    Vselection_coding_system : Vnext_selection_coding_system;
+  if (!NILP (cached_coding_system)
+      && EQ (cached_coding_system, current_coding_system))
+    return;
+  cached_coding_system = current_coding_system;
+  
+  /* Set some sensible fallbacks */
+  codepage = ANSICP;
+  lcid = LOCALE_NEUTRAL;
+  clipboard_type = CF_TEXT;
+
+  /* Interpret the coding system symbol name */
+  coding_name = SDATA (SYMBOL_NAME (current_coding_system));
+
+  /* "(.*-)?utf-16.*" -> CF_UNICODETEXT */
+  cp = strstr (coding_name, "utf-16");
+  if (cp != NULL && (cp == coding_name || cp[-1] == '-'))
+    {
+      clipboard_type = CF_UNICODETEXT;
+      return;
+    }
+
+  /* "cp[0-9]+.*" or "windows-[0-9]+.*" -> CF_TEXT or CF_OEMTEXT */
+  slen = strlen (coding_name);
+  if (slen >= 4 && coding_name[0] == 'c' && coding_name[1] == 'p')
+    cp = coding_name + 2;
+  else if (slen >= 10 && memcmp (coding_name, "windows-", 8) == 0)
+    cp = coding_name + 8;
+  else
+    return;
+
+  end = (char*)cp;
+  codepage = strtol (cp, &end, 10);
+
+  /* Error return from strtol() or number of digits < 2 -> Restore the
+     default and drop it. */
+  if (codepage == 0 || (end-cp) < 2 )
+    {
+      codepage = ANSICP;
+      return;
+    }
+
+  /* Is it the current system default? */
+  if (codepage == ANSICP)
+    {
+      /* clipboard_type = CF_TEXT; */
+      return;
+    }
+  if (codepage == OEMCP)
+    {
+      clipboard_type = CF_OEMTEXT;
+      return;
+    }
+
+  /* Else determine a suitable locale the hard way. */
+  EnumSystemLocales (enum_locale_callback, LCID_INSTALLED);
+}
+
 DEFUN ("w32-set-clipboard-data", Fw32_set_clipboard_data,
        Sw32_set_clipboard_data, 1, 2, 0,
        doc: /* This sets the clipboard data to the given text.  */)
@@ -117,147 +267,176 @@ DEFUN ("w32-set-clipboard-data", Fw32_se
     Lisp_Object string, frame;
 {
   BOOL ok = TRUE;
-  HANDLE htext;
+  HANDLE htext, hlocale = NULL;
   int nbytes;
   int truelen, nlines = 0;
+  int require_encoding = 0;
   unsigned char *src;
   unsigned char *dst;
+  unsigned char *end;
+  UINT actual_clipboard_type;
 
   CHECK_STRING (string);
 
   if (!NILP (frame))
     CHECK_LIVE_FRAME (frame);
 
+  setup_parameters ();
+  actual_clipboard_type = clipboard_type;
+  
   BLOCK_INPUT;
 
   nbytes = SBYTES (string) + 1;
   src = SDATA (string);
-  dst = src;
 
-  /* We need to know how many lines there are, since we need CRLF line
-     termination for compatibility with other Windows Programs.
-     avoid using strchr because it recomputes the length every time */
-  while ((dst = memchr (dst, '\n', nbytes - (dst - src))) != NULL)
+  /* Check for non-ASCII characters.  While we are at it, count the
+     number of LFs, so we know how many CRs we have to add later
+     on.  */
+  for (dst = src, end = src+nbytes; dst < end; dst++)
     {
-      nlines++;
-      dst++;
+      if (*dst == '\n')
+	nlines++;
+      else if (*dst >= 0x80 || *dst == 0)
+	{
+	  require_encoding = 1;
+	  break;
+	}
     }
 
-  {
-    /* Since we are now handling multilingual text, we must consider
-       encoding text for the clipboard.  */
-    int charset_info = find_charset_in_text (src, SCHARS (string),
-					     nbytes, NULL, Qnil);
-
-    if (charset_info == 0)
-      {
-	/* No multibyte character in OBJ.  We need not encode it.  */
+  if (!require_encoding)
+    {
+      /* No multibyte character in OBJ.  We need not encode it.  */
 
-	/* Need to know final size after CR chars are inserted (the
-	   standard CF_TEXT clipboard format uses CRLF line endings,
-	   while Emacs uses just LF internally).  */
+      /* Need to know final size after CR chars are inserted (the
+	 standard CF_TEXT clipboard format uses CRLF line endings,
+	 while Emacs uses just LF internally).  */
 
-	truelen = nbytes + nlines;
+      truelen = nbytes + nlines;
 
-	if ((htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, truelen)) == NULL)
+      if ((htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, truelen)) == NULL)
 	  goto error;
 
-	if ((dst = (unsigned char *) GlobalLock (htext)) == NULL)
-	  goto error;
+      if ((dst = (unsigned char *) GlobalLock (htext)) == NULL)
+	goto error;
 
-	/* convert to CRLF line endings expected by clipboard */
-	while (1)
-	  {
-	    unsigned char *next;
-	    /* copy next line or remaining bytes including '\0' */
-	    next = _memccpy (dst, src, '\n', nbytes);
-	    if (next)
-	      {
-		/* copied one line ending with '\n' */
-		int copied = next - dst;
-		nbytes -= copied;
-		src += copied;
-		/* insert '\r' before '\n' */
-		next[-1] = '\r';
-		next[0] = '\n';
-		dst = next + 1;
-	      }
-	    else
-	      /* copied remaining partial line -> now finished */
-	      break;
-	  }
+      /* convert to CRLF line endings expected by clipboard */
+      while (1)
+	{
+	  unsigned char *next;
+	  /* copy next line or remaining bytes including '\0' */
+	  next = _memccpy (dst, src, '\n', nbytes);
+	  if (next)
+	    {
+	      /* copied one line ending with '\n' */
+	      int copied = next - dst;
+	      nbytes -= copied;
+	      src += copied;
+	      /* insert '\r' before '\n' */
+	      next[-1] = '\r';
+	      next[0] = '\n';
+	      dst = next + 1;
+	    }
+	  else
+	    /* copied remaining partial line -> now finished */
+	    break;
+	}
 
-	GlobalUnlock (htext);
+      GlobalUnlock (htext);
 
-	Vlast_coding_system_used = Qraw_text;
-      }
-    else
-      {
-	/* We must encode contents of OBJ to the selection coding
-           system. */
-	int bufsize;
-	struct coding_system coding;
-	HANDLE htext2;
+      /* Truncate last_clipboard_text by setting it to the string
+	 terminator. */
+      if (last_clipboard_text)
+	last_clipboard_text[0] = last_clipboard_text[1] = '\0';
 
-	if (NILP (Vnext_selection_coding_system))
-	  Vnext_selection_coding_system = Vselection_coding_system;
-	setup_coding_system
-	  (Fcheck_coding_system (Vnext_selection_coding_system), &coding);
-	if (SYMBOLP (coding.pre_write_conversion)
-	    && !NILP (Ffboundp (coding.pre_write_conversion)))
-	  {
-	    string = run_pre_post_conversion_on_str (string, &coding, 1);
-	    src = SDATA (string);
-	    nbytes = SBYTES (string);
-	  }
-	coding.src_multibyte = 1;
-	coding.dst_multibyte = 0;
-	/* Need to set COMPOSITION_DISABLED, otherwise Emacs crashes in
-	   encode_coding_iso2022 trying to dereference a null pointer.  */
-	coding.composing = COMPOSITION_DISABLED;
-	if (coding.type == coding_type_iso2022)
-	  coding.flags |= CODING_FLAG_ISO_SAFE;
-	Vnext_selection_coding_system = Qnil;
-	coding.mode |= CODING_MODE_LAST_BLOCK;
-	bufsize = encoding_buffer_size (&coding, nbytes);
-	if ((htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, bufsize)) == NULL)
-	  goto error;
-	if ((dst = (unsigned char *) GlobalLock (htext)) == NULL)
-	  goto error;
-	encode_coding (&coding, src, dst, nbytes, bufsize);
-	Vlast_coding_system_used = coding.symbol;
+      Vlast_coding_system_used = Qraw_text;
+      actual_clipboard_type = CF_TEXT;
+    }
+  else
+    {
+      /* We must encode contents of OBJ to the selection coding
+	 system. */
+      int bufsize;
+      struct coding_system coding;
+      HANDLE htext2;
+
+      if (NILP (Vnext_selection_coding_system))
+	Vnext_selection_coding_system = Vselection_coding_system;
+      setup_coding_system
+	(Fcheck_coding_system (Vnext_selection_coding_system), &coding);
+      if (SYMBOLP (coding.pre_write_conversion)
+	  && !NILP (Ffboundp (coding.pre_write_conversion)))
+	{
+	  string = run_pre_post_conversion_on_str (string, &coding, 1);
+	  src = SDATA (string);
+	  nbytes = SBYTES (string);
+	}
+      coding.src_multibyte = 1;
+      coding.dst_multibyte = 0;
+      /* Need to set COMPOSITION_DISABLED, otherwise Emacs crashes in
+	 encode_coding_iso2022 trying to dereference a null pointer.  */
+      coding.composing = COMPOSITION_DISABLED;
+      if (coding.type == coding_type_iso2022)
+	coding.flags |= CODING_FLAG_ISO_SAFE;
+      Vnext_selection_coding_system = Qnil;
+      coding.mode |= CODING_MODE_LAST_BLOCK;
+      bufsize = encoding_buffer_size (&coding, nbytes) +2;
+      if ((htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, bufsize)) == NULL)
+	goto error;
+      if ((dst = (unsigned char *) GlobalLock (htext)) == NULL)
+	goto error;
+
+      encode_coding (&coding, src, dst, nbytes, bufsize-2);
+      Vlast_coding_system_used = coding.symbol;
+
+      /* Add the string terminator. */
+      dst[coding.produced] = dst[coding.produced+1] = '\0';
+
+      /* If clipboard sequence numbers are not supported, keep a copy for
+	 later comparison.  */
+      if (!clipboard_sequence_fn)
+	{
+	  /* Stash away the data we are about to put into the
+	     clipboard, so we could later check inside
+	     Fw32_get_clipboard_data whether the clipboard still
+	     holds our data.  */
+	  if (clipboard_storage_size < coding.produced+2)
+	    {
+	      clipboard_storage_size = coding.produced + 100;
+	      last_clipboard_text = (char *) xrealloc (last_clipboard_text,
+						       clipboard_storage_size);
+	    }
+	  if (last_clipboard_text)
+	      memcpy (last_clipboard_text, dst, coding.produced+2);
+	}
 
-	/* If clipboard sequence numbers are not supported, keep a copy for
-	   later comparison.  */
-	if (!clipboard_sequence_fn)
-	  {
-	    /* Stash away the data we are about to put into the
-	       clipboard, so we could later check inside
-	       Fw32_get_clipboard_data whether the clipboard still
-	       holds our data.  */
-	    if (clipboard_storage_size < coding.produced)
-	      {
-		clipboard_storage_size = coding.produced + 100;
-		last_clipboard_text = (char *) xrealloc (last_clipboard_text,
-							 clipboard_storage_size);
-	      }
-	    if (last_clipboard_text)
-	      memcpy (last_clipboard_text, dst, coding.produced);
-	  }
+      GlobalUnlock (htext);
 
-	GlobalUnlock (htext);
+      /* Shrink data block to actual size.  */
+      htext2 = GlobalReAlloc (htext, coding.produced+2,
+			      GMEM_MOVEABLE | GMEM_DDESHARE);
+      if (htext2 != NULL) htext = htext2;
 
-	/* Shrink data block to actual size.  */
-	htext2 = GlobalReAlloc (htext, coding.produced,
-                                GMEM_MOVEABLE | GMEM_DDESHARE);
-	if (htext2 != NULL) htext = htext2;
-      }
-  }
+      if (lcid != LOCALE_NEUTRAL && lcid != DEFAULT_LCID)
+	{
+	  hlocale = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, sizeof (lcid));
+	  if (hlocale != NULL)
+	    {
+	      * (LCID*) GlobalLock (hlocale) = lcid;
+	      GlobalUnlock (hlocale);
+	    }
+	}
+    }
 
   if (!OpenClipboard ((!NILP (frame) && FRAME_W32_P (XFRAME (frame))) ? FRAME_W32_WINDOW (XFRAME (frame)) : NULL))
     goto error;
 
-  ok = EmptyClipboard () && SetClipboardData (CF_TEXT, htext);
+  ok = EmptyClipboard () && SetClipboardData (actual_clipboard_type, htext);
+
+  if (ok && hlocale != NULL)
+    {
+      ok = ok && SetClipboardData (CF_LOCALE, hlocale);
+      if (!ok) EmptyClipboard ();
+    }
 
   CloseClipboard ();
 
@@ -277,8 +456,12 @@ DEFUN ("w32-set-clipboard-data", Fw32_se
 
   ok = FALSE;
   if (htext) GlobalFree (htext);
+  if (hlocale) GlobalFree (hlocale);
+
+  /* Truncate last_clipboard_text by setting it to the string
+     terminator. */
   if (last_clipboard_text)
-    *last_clipboard_text = '\0';
+    last_clipboard_text[0] = last_clipboard_text[1] = '\0';
 
   last_clipboard_sequence_number = 0;
 
@@ -296,16 +479,40 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
 {
   HANDLE htext;
   Lisp_Object ret = Qnil;
+  UINT actual_clipboard_type;
+  int use_configured_coding_system = 1;
 
   if (!NILP (frame))
     CHECK_LIVE_FRAME (frame);
 
+  setup_parameters ();
+  actual_clipboard_type = clipboard_type;
+
   BLOCK_INPUT;
 
   if (!OpenClipboard ((!NILP (frame) && FRAME_W32_P (XFRAME (frame))) ? FRAME_W32_WINDOW (XFRAME (frame)) : NULL))
     goto done;
 
-  if ((htext = GetClipboardData (CF_TEXT)) == NULL)
+  if ((htext = GetClipboardData (clipboard_type)) == NULL)
+    {
+      /* If we want CF_UNICODETEXT but can't get it, the current
+	 coding system is useless.  OTOH we can still try and decode
+	 CF_TEXT based on the locale that the system gives us and that
+	 we get down below.  Note that Windows is documented to always
+	 generate CF_LOCALE info automatically, so the locale handle
+	 should be always present (actually this is not always true on
+	 W'9x ;-().  */
+      if (clipboard_type == CF_UNICODETEXT)
+	{
+	  htext = GetClipboardData (CF_TEXT);
+	  if (htext != NULL)
+	    {
+	      actual_clipboard_type = CF_TEXT;
+	      use_configured_coding_system = 0;
+	    }
+	}
+    }
+  if (htext == NULL)
     goto closeclip;
 
   {
@@ -318,7 +525,10 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
     if ((src = (unsigned char *) GlobalLock (htext)) == NULL)
       goto closeclip;
 
-    nbytes = strlen (src);
+    if (actual_clipboard_type == CF_UNICODETEXT)
+      nbytes = (lstrlenW ((WCHAR *)src) + 1) * 2;
+    else
+      nbytes = strlen (src) + 1;
 
     /* If the text in clipboard is identical to what we put there
        last time w32_set_clipboard_data was called, pretend there's no
@@ -329,34 +539,99 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
 	 && clipboard_sequence_fn () == last_clipboard_sequence_number)
 	|| (last_clipboard_text
 	    && clipboard_storage_size >= nbytes
-	    && memcmp(last_clipboard_text, src, nbytes) == 0))
+	    && memcmp (last_clipboard_text, src, nbytes) == 0))
       goto closeclip;
 
-    {
-      /* If the clipboard data contains any non-ascii code, we
-	 need to decode it.  */
-      int i;
+    /* Drop the string terminator from here on. */
+    nbytes -= actual_clipboard_type == CF_UNICODETEXT ? 2 : 1;
 
-      for (i = 0; i < nbytes; i++)
-	{
-	  if (src[i] >= 0x80)
-	    {
-	      require_decoding = 1;
-	      break;
-	    }
-	}
-    }
+    /* If the clipboard data contains any non-ascii code, we need to
+       decode it with a coding system.
+       FIXME: Repeat the code for the Unicode case. */
+    if (actual_clipboard_type == CF_UNICODETEXT)
+      require_decoding = 1;
+    else
+      {
+	int i;
+
+	for (i = 0; i < nbytes; i++)
+	  {
+	    if (src[i] >= 0x80)
+	      {
+		require_decoding = 1;
+		break;
+	      }
+	  }
+      }
 
     if (require_decoding)
       {
 	int bufsize;
 	unsigned char *buf;
 	struct coding_system coding;
+	Lisp_Object coding_system = Qnil;
+	
+	/* `next-selection-coding-system' should override everything,
+	   even when the locale passed by the system disagrees.  The
+	   only exception is when `next-selection-coding-system'
+	   requested CF_UNICODETEXT and we couldn't get that. */
+	if (use_configured_coding_system
+	    && !NILP (Vnext_selection_coding_system))
+	    coding_system = Vnext_selection_coding_system;
+
+	/* If we have CF_TEXT or CF_OEMTEXT, we want to check out
+	   CF_LOCALE, too. */
+	else if (actual_clipboard_type != CF_UNICODETEXT)
+	  {
+	    HANDLE hlocale;
+	    LCID cb_lcid;
+	    UINT cp;
+
+	    hlocale = GetClipboardData (CF_LOCALE);
+	    if (hlocale != NULL)
+	      {
+		cb_lcid = *(LCID*)GlobalLock (hlocale);
+		GlobalUnlock (hlocale);
 
-	if (NILP (Vnext_selection_coding_system))
-	  Vnext_selection_coding_system = Vselection_coding_system;
-	setup_coding_system
-	  (Fcheck_coding_system (Vnext_selection_coding_system), &coding);
+		/* W'9x has garbage as the sort order (i.e. there is
+		   another instance of the language id in the upper
+		   word).  We can just filter out the unneeded
+		   mis-information to avoid irritations. */
+		cb_lcid = MAKELCID (LANGIDFROMLCID (cb_lcid), SORT_DEFAULT);
+	      }
+	    else
+	      cb_lcid = DEFAULT_LCID;
+
+	    /* If we are using fallback from CF_UNICODETEXT, we can't
+	       use the configured coding system.  Also we don't want
+	       to use it, if the system has supplied us with a locale
+	       and it is not just the system default. */
+	    if (!use_configured_coding_system || cb_lcid != DEFAULT_LCID)
+	      {
+		cp = cp_from_locale (cb_lcid,
+				     actual_clipboard_type == CF_TEXT
+				     ? LOCALE_IDEFAULTANSICODEPAGE
+				     : LOCALE_IDEFAULTCODEPAGE);
+		/* If it's just our current setting anyway, use the
+		   coding system that the user has selected.
+		   Otherwise create a new spec to match the locale
+		   that was specified by the other side or the
+		   system. */
+		if (!use_configured_coding_system || cp != codepage)
+		  {
+		    char buffer[30];
+		    sprintf (buffer, "cp%d-dos", (int) cp);
+		    coding_system = intern (buffer);
+		    /* FIXME: We should verify that this coding system
+		       actually exists. */
+		  }
+	      }
+	  }
+
+	if (NILP (coding_system))
+	  coding_system = Vselection_coding_system;
+
+	setup_coding_system (Fcheck_coding_system (coding_system), &coding);
 	coding.src_multibyte = 0;
 	coding.dst_multibyte = 1;
 	Vnext_selection_coding_system = Qnil;
@@ -459,8 +734,12 @@ and t is the same as `SECONDARY'.  */)
       if (OpenClipboard (NULL))
 	{
 	  int format = 0;
+	  setup_parameters ();
 	  while (format = EnumClipboardFormats (format))
-	    if (format == CF_TEXT)
+	    /* Check CF_TEXT in addition to clipboard_type, because we
+	       fall back on that if CF_UNICODETEXT is not
+	       available.  */
+	    if (format == clipboard_type || format == CF_TEXT)
 	      {
 		val = Qt;
 		break;
@@ -500,6 +779,8 @@ set to nil.  */);
   Vnext_selection_coding_system = Qnil;
 
   QCLIPBOARD = intern ("CLIPBOARD");	staticpro (&QCLIPBOARD);
+
+  cached_coding_system = Qnil; staticpro (&cached_coding_system);
 }
 
 /* arch-tag: c96e9724-5eb1-4dad-be07-289f092fd2af

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Unicode support for the MS Windows clipboard

[Kenichi Handa]

In article <m3u0xtdmqe.fsf@seneca.benny.turtle-trading.net>, Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:

> The C code constructs it's own cpXXXX-dos coding system
> names in one place.  I would like to check that the
> constructed coding system actually exists, but I don't
> know how yet.

> I didn't worry about the console encoding (CF_OEMTEXT) too
> much, but I added support for it, where it seemed
> straight-forward to me.

> I get the feeling that the C code does too much, and that
> it should delegate major parts of the processing to Lisp.
> I am not yet sure how to structure that, though.

As I've never read w32select.c, the following may be out of
point, but I'll briefly explain how X selection is handled.

All decoding and encoding of X selection is done in
select.el.

For reading a selection data:

The entry point is x-get-selection.

It calls x-get-selection-internal of xselect.c.

It at first tries to get a locale selection value
(i.e. Emacs is the owner of the selection) by
calling x_get_local_selection with `local_request' 1..

If that fails, it tries to get a selection from some other
client by x_get_foreign_selection.  This function receives a
selection data and convert it to a Lispy data via
selection_data_to_lisp_data.  This function generate a
unibyte string and put `foreign-selection' property on it.

After a string is returned to x-get-selection, it checks
`foreign-selection' property and if it is on, decode the
unibyte string according to data-type and
selection-coding-system.


For sending a selection data:

The entry point is x_handle_selection_request of xselect.c
which is called from swallow_events of keyboard.c when Emacs
gets SELECTION_REQUEST_EVENT.

x_handle_selection_request calls x_get_local_selection too
but with `local_request' 0.  If Emacs owns a selection data,
it calls one of Lisp handler registered in
selection-converter-alist.  Typical case is
xselect-convert-to-string of select.el called with TYPE
non-nil.  This function does all encoding work (if TYPE is
non-nil) and return an encoded unibyte string.

---
Ken'ichi HANDA
handa@m17n.org

Re: Unicode support for the MS Windows clipboard

[Eli Zaretskii]

> From: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de>
> Date: Thu, 03 Jun 2004 11:17:29 +0200
> 
> The C code constructs it's own cpXXXX-dos coding system names in one
> place.  I would like to check that the constructed coding system
> actually exists, but I don't know how yet.

The function coding-system-p should allow you to do that.

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

[-- Attachment #1: Type: text/plain, Size: 4814 bytes --]

Hi all,


Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:
> While I am still familiar with the internal of this module, I'd like
> to tackle delayed rendering.

Done that, see attached expanded patch. 

I consider this done from a functional POV, but of course feel free to
suggest changes.



Questions:

Wrapping the Lisp evaluation and en/de-coding in appropriate error
handlers for asynchronous execution is a bit of a pain.  Is there
documentation how this *should* work or was this kind of scenario just
never considered before?

It seems that the GC macros are no-ops on Windows?  If so these should
be removed, I guess.  If not, other variables may need the same
treatment.

w32.c has a note that atexit() is broken.  atexit() works here with
Mingw.  The note in w32.c is probably about a static build, which
Mingw doesn't support.  Anyway, the alternative implementation using
signal(SIGABRT) as in w32.c doesn't work here.



About my earlier notes and discussion:

> I am thinking of constructing a hidden window for hanging the
> message handlers on [...]

That's what I have done here.  The FRAME parameter for the Lisp
functions is ignored now.  This parameter is never passed in the core
code Lisp anyway, I believe.  If the change in interface is
acceptable, the parameter could be removed altogether.

> The C code constructs it's own cpXXXX-dos coding system names in one
> place.  I would like to check that the constructed coding system
> actually exists, but I don't know how yet.

That check is actually made by the code that uses the coding system.
I had missed that before.

> I get the feeling that the C code does too much, and that it should
> delegate major parts of the processing to Lisp.  I am not yet sure
> how to structure that, though.

After thinking about it, making more interfaces between Lisp and C
doesn't seem too usefull.  Windows is pretty firm about what is what
here, so there is not much to customize.  Via the
selection-coding-system variable, the algorithm is already more
tinkerable from the Lisp side than strictly necessary.

> Somebody suggested to automatically map all Unicode coding-systems
> to the right one, utf-16le-dos.

The uncertainty about the right Unicode coding-system to use seems to
me to be a general problem with the naming of the Unicode
coding-systems, not with w32select.c.  The names of the coding-systems
could be clearer.  At the moment (in CVS) we have "utf-16le" which
acts like be "utf-16-le-without-signature" and "utf-16-le" which is
the same as "utf-16-le-with-signature".

OTOH maybe *appropriate* aliases could be added to make some
selections more obvious.  E.g. on Windows we might want a simple alias
"unicode" for what is now "utf-16le", because that is what Windows
users and programmers expect when they hear "Unicode" encoding.

> I didn't do anything about the defaults yet.

I put the dynamic initializations into the C code now and dropped the
line of Lisp code that sets this in mule-cmds.el.  This was mostly
because I couldn't find Lisp variables on which to base the decisions.
It also keeps things together this way.



benny


(Note: I can of course shorten the ChangeLog, if necessary.)

2004-07-26  Benjamin Riefenstahl  <Benjamin.Riefenstahl@epost.de>

	* w32select.c: Drop last_clipboard_text and related code, keep
        track of ownership via clipboard_owner instead.  Drop old #if0
        sections.

	(DEFAULT_LCID, ANSICP, OEMCP, QUNICODE, QANSICP, QOEMCP)
	(clipboard_owner, modifying_clipboard, cfg_coding_system)
	(cfg_codepage, cfg_lcid, cfg_clipboard_type, current_text)
	(current_coding_system, current_requires_encoding)
	(current_num_nls, current_clipboard_type, current_lcid): New
	static variables.

	(convert_to_handle_as_ascii, convert_to_handle_as_coded)
        (render, render_all, run_protected, lisp_error_handler)
	(owner_callback, create_owner, atexit_callback, setup_config)
	(enum_locale_callback, cp_from_locale, coding_from_cp)
        (globals_of_w32select): New local functions.

	(Fw32_set_clipboard_data): Ignore parameter FRAME, use
	clipboard_owner instead.  Use delayed rendering and provide
	all text formats.  Provide CF_LOCALE if necessary.

	(Fw32_get_clipboard_data): Handle CF_UNICODETEXT and
	CF_LOCALE.  Fall back to CF_TEXT, if CF_UNICODETEXT is not
	available.  Force DOS line-ends for decoding. 

	(Fx_selection_exists_p): Handle CF_UNICODETEXT.

	(syms_of_w32select): Init and register new variables.

        * w32.h: Add prototype for globals_of_w32select.  Make the
        neighboring K&R declarations into prototypes, too.

        * emacs.c: Include w32.h to get function prototypes.
        (main): Call globals_of_w32select.

        * mule-cmds.el (set-locale-environment): Remove call to
        set-selection-coding-system on Windows.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: w32select-2.patch --]
[-- Type: text/x-patch, Size: 39623 bytes --]

Index: lisp/international/mule-cmds.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/international/mule-cmds.el,v
retrieving revision 1.257
diff -u -p -r1.257 mule-cmds.el
--- lisp/international/mule-cmds.el	12 Jun 2004 02:21:21 -0000	1.257
+++ lisp/international/mule-cmds.el	26 Jul 2004 18:33:01 -0000
@@ -2422,13 +2422,13 @@ See also `locale-charset-language-names'
 	  (prefer-coding-system coding-system)
 	  (setq locale-coding-system coding-system))))
 
-    ;; On Windows, override locale-coding-system, keyboard-coding-system,
-    ;; selection-coding-system with system codepage.
+    ;; On Windows, override locale-coding-system,
+    ;; keyboard-coding-system, with system codepage.  Note:
+    ;; selection-coding-system is already set in w32select.c.
     (when (boundp 'w32-ansi-code-page)
       (let ((code-page-coding (intern (format "cp%d" w32-ansi-code-page))))
 	(when (coding-system-p code-page-coding)
 	  (setq locale-coding-system code-page-coding)
-	  (set-selection-coding-system code-page-coding)
 	  (set-keyboard-coding-system code-page-coding)
 	  (set-terminal-coding-system code-page-coding))))
 
Index: src/emacs.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/emacs.c,v
retrieving revision 1.342
diff -u -p -r1.342 emacs.c
--- src/emacs.c	24 Jun 2004 20:24:52 -0000	1.342
+++ src/emacs.c	26 Jul 2004 18:33:17 -0000
@@ -42,6 +42,8 @@ Boston, MA 02111-1307, USA.  */
 
 #ifdef WINDOWSNT
 #include <fcntl.h>
+#include <windows.h> /* just for w32.h */
+#include "w32.h"
 #endif
 
 #include "lisp.h"
@@ -1578,6 +1580,7 @@ main (argc, argv
 #ifdef HAVE_NTGUI
       globals_of_w32fns ();
       globals_of_w32menu ();
+      globals_of_w32select ();
 #endif  /* end #ifdef HAVE_NTGUI */
     }
 
Index: src/w32.h
===================================================================
RCS file: /cvsroot/emacs/emacs/src/w32.h,v
retrieving revision 1.15
diff -u -p -r1.15 w32.h
--- src/w32.h	1 Sep 2003 15:45:57 -0000	1.15
+++ src/w32.h	26 Jul 2004 18:33:24 -0000
@@ -122,16 +122,17 @@ extern void reset_standard_handles (int 
 /* Return the string resource associated with KEY of type TYPE.  */
 extern LPBYTE w32_get_resource (char * key, LPDWORD type);
 
-extern void init_ntproc ();
-extern void term_ntproc ();
-extern void globals_of_w32 ();
-extern void syms_of_w32term ();
-extern void syms_of_w32fns ();
-extern void globals_of_w32fns ();
-extern void syms_of_w32select ();
-extern void syms_of_w32menu ();
-extern void globals_of_w32menu ();
-extern void syms_of_fontset ();
+extern void init_ntproc (void);
+extern void term_ntproc (void);
+extern void globals_of_w32 (void);
+extern void syms_of_w32term (void);
+extern void syms_of_w32fns (void);
+extern void globals_of_w32fns (void);
+extern void syms_of_w32select (void);
+extern void globals_of_w32select (void);
+extern void syms_of_w32menu (void);
+extern void globals_of_w32menu (void);
+extern void syms_of_fontset (void);
 
 #endif /* EMACS_W32_H */
 
Index: src/w32select.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/w32select.c,v
retrieving revision 1.33
diff -u -p -r1.33 w32select.c
--- src/w32select.c	30 Jun 2004 23:30:39 -0000	1.33
+++ src/w32select.c	26 Jul 2004 18:33:24 -0000
@@ -1,5 +1,5 @@
 /* Selection processing for Emacs on the Microsoft W32 API.
-   Copyright (C) 1993, 1994 Free Software Foundation.
+   Copyright (C) 1993, 1994, 2004 Free Software Foundation.
 
 This file is part of GNU Emacs.
 
@@ -18,273 +18,672 @@ along with GNU Emacs; see the file COPYI
 the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
 Boston, MA 02111-1307, USA.  */
 
-/* Written by Kevin Gallo */
+/* Written by Kevin Gallo, Benjamin Riefenstahl */
 
 #include <config.h>
 #include "lisp.h"
 #include "w32term.h"	/* for all of the w32 includes */
-#include "dispextern.h"	/* frame.h seems to want this */
-#include "keyboard.h"
-#include "frame.h"	/* Need this to get the X window of selected_frame */
+#include "w32heap.h"	/* os_subtype */
 #include "blockinput.h"
-#include "buffer.h"
+#include "keyboard.h"	/* cmd_error_internal() */
 #include "charset.h"
 #include "coding.h"
 #include "composite.h"
 
+
+static HGLOBAL convert_to_handle_as_ascii (void);
+static HGLOBAL convert_to_handle_as_coded (Lisp_Object coding_system);
+static Lisp_Object render (Lisp_Object oformat);
+static Lisp_Object render_all (void);
+static void run_protected (Lisp_Object (*code) (), Lisp_Object arg);
+static Lisp_Object lisp_error_handler (Lisp_Object error);
+static LRESULT CALLBACK owner_callback (HWND win, UINT msg,
+					WPARAM wp, LPARAM lp);
+static HWND create_owner (void);
+static void atexit_callback (void);
+
+static void setup_config (void);
+static BOOL WINAPI enum_locale_callback (/*const*/ char* loc_string);
+static UINT cp_from_locale (LCID lcid, UINT format);
+static Lisp_Object coding_from_cp (UINT codepage);
+
+
 Lisp_Object QCLIPBOARD;
 
 /* Coding system for communicating with other Windows programs via the
    clipboard.  */
 static Lisp_Object Vselection_coding_system;
 
-/* Coding system for the next communicating with other Windows programs.  */
+/* Coding system for the next communication with other Windows
+   programs.  */
 static Lisp_Object Vnext_selection_coding_system;
 
-/* Sequence number, used where possible to detect when we are pasting
-   our own text.  */
-static DWORD last_clipboard_sequence_number;
-extern ClipboardSequence_Proc clipboard_sequence_fn;
-
-/* The last text we put into the clipboard.  This is used when the OS
-   does not support sequence numbers (NT4, 95). It is undesirable to
-   use data put on the clipboard by Emacs because the clipboard data
-   could be MULEtilated by inappropriately chosen
-   (next-)selection-coding-system.  For this reason, we must store the
-   text *after* it was encoded/Unix-to-DOS-converted.  */
-static unsigned char *last_clipboard_text = NULL;
-static size_t clipboard_storage_size = 0;
-
-#if 0
-DEFUN ("w32-open-clipboard", Fw32_open_clipboard, Sw32_open_clipboard, 0, 1, 0,
-       doc: /* This opens the clipboard with the given frame pointer.  */)
-     (frame)
-     Lisp_Object frame;
+/* Internal pseudo-constants, initialized in globals_of_w32select()
+   based on current system parameters. */
+static LCID DEFAULT_LCID;
+static UINT ANSICP, OEMCP;
+static Lisp_Object QUNICODE, QANSICP, QOEMCP;
+
+/* A hidden window just for the clipboard management. */
+static HWND clipboard_owner;
+/* A flag to tell WM_DESTROYCLIPBOARD who is to blame this time (just
+   checking GetClipboardOwner() doesn't work, sadly). */
+static int modifying_clipboard = 0;
+
+/* Configured transfer parameters, based on the last inspection of
+   selection-coding-system.  */
+static Lisp_Object cfg_coding_system;
+static UINT cfg_codepage;
+static LCID cfg_lcid;
+static UINT cfg_clipboard_type;
+
+/* The current state for delayed rendering. */
+static Lisp_Object current_text;
+static Lisp_Object current_coding_system;
+static int current_requires_encoding, current_num_nls;
+static UINT current_clipboard_type;
+static LCID current_lcid;
+
+#if TRACE
+#define ONTRACE(stmt) stmt
+#else
+#define ONTRACE(stmt) /*stmt*/
+#endif
+
+
+/* This function assumes that there is no multibyte character in
+   current_text, so we can short-cut encoding.  */
+
+static HGLOBAL
+convert_to_handle_as_ascii (void)
 {
-  BOOL ok = FALSE;
+  HGLOBAL htext = NULL;
+  int nbytes;
+  int truelen;
+  unsigned char *src;
+  unsigned char *dst;
 
-  if (!NILP (frame))
-    CHECK_LIVE_FRAME (frame);
+  ONTRACE (fprintf (stderr, "convert_to_handle_as_ascii\n"));
 
-  BLOCK_INPUT;
+  nbytes = SBYTES (current_text) + 1;
+  src = SDATA (current_text);
 
-  ok = OpenClipboard ((!NILP (frame) && FRAME_W32_P (XFRAME (frame))) ? FRAME_W32_WINDOW (XFRAME (frame)) : NULL);
+  /* We need to add to the size the number of LF chars where we have
+     to insert CR chars (the standard CF_TEXT clipboard format uses
+     CRLF line endings, while Emacs uses just LF internally).  */
 
-  UNBLOCK_INPUT;
+  truelen = nbytes + current_num_nls;
+
+  if ((htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, truelen)) == NULL)
+    return NULL;
+
+  if ((dst = (unsigned char *) GlobalLock (htext)) == NULL)
+    {
+      GlobalFree (htext);
+      return NULL;
+    }
+
+  /* convert to CRLF line endings expected by clipboard */
+  while (1)
+    {
+      unsigned char *next;
+      /* copy next line or remaining bytes including '\0' */
+      next = _memccpy (dst, src, '\n', nbytes);
+      if (next)
+	{
+	  /* copied one line ending with '\n' */
+	  int copied = next - dst;
+	  nbytes -= copied;
+	  src += copied;
+	  /* insert '\r' before '\n' */
+	  next[-1] = '\r';
+	  next[0] = '\n';
+	  dst = next + 1;
+	}
+      else
+	/* copied remaining partial line -> now finished */
+	break;
+    }
 
-  return (ok ? frame : Qnil);
+  GlobalUnlock (htext);
+
+  return htext;
 }
 
-DEFUN ("w32-empty-clipboard", Fw32_empty_clipboard,
-       Sw32_empty_clipboard, 0, 0, 0,
-       doc: /* Empty the clipboard.
-Assigns ownership of the clipboard to the window which opened it.  */)
-     ()
+/* This function assumes that there are multibyte or NUL characters in
+   current_text, or that we need to construct Unicode.  It runs the
+   text through the encoding machinery.  */
+
+static HGLOBAL
+convert_to_handle_as_coded (Lisp_Object coding_system)
 {
-  BOOL ok = FALSE;
+  HGLOBAL htext = NULL, htext2;
+  int nbytes;
+  unsigned char *src;
+  unsigned char *dst = NULL;
+  int bufsize;
+  struct coding_system coding;
+  Lisp_Object string = Qnil;
+  struct gcpro gcpro1;
+
+  ONTRACE (fprintf (stderr, "convert_to_handle_as_coded: %s\n",	
+		    SDATA (SYMBOL_NAME (coding_system))));
+
+  /* FIXME: It looks like GC_MARK_STACK == GC_MAKE_GCPROS_NOOPS on
+     Windows (see src/lisp.h and nt/config.nt), so this is probably
+     unnecessary.  If it was necessary, we'd probably have to protect
+     all the symbols, too. */
+  GCPRO1 (string);
+
+  setup_coding_system (Fcheck_coding_system (coding_system), &coding);
+  coding.src_multibyte = 1;
+  coding.dst_multibyte = 0;
+  /* Need to set COMPOSITION_DISABLED, otherwise Emacs crashes in
+     encode_coding_iso2022 trying to dereference a null pointer.  */
+  coding.composing = COMPOSITION_DISABLED;
+  if (coding.type == coding_type_iso2022)
+    coding.flags |= CODING_FLAG_ISO_SAFE;
+  coding.mode |= CODING_MODE_LAST_BLOCK;
+  /* Force DOS line-ends. */
+  coding.eol_type = CODING_EOL_CRLF;
+
+  if (SYMBOLP (coding.pre_write_conversion)
+      && !NILP (Ffboundp (coding.pre_write_conversion)))
+    string = run_pre_post_conversion_on_str (current_text, &coding, 1);
+  else
+    string = current_text;
 
-  BLOCK_INPUT;
+  nbytes = SBYTES (string);
+  src = SDATA (string);
 
-  ok = EmptyClipboard ();
+  bufsize = encoding_buffer_size (&coding, nbytes) +2;
+  htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, bufsize);
 
-  UNBLOCK_INPUT;
+  if (htext != NULL)
+    dst = (unsigned char *) GlobalLock (htext);
+
+  if (dst != NULL)
+    {
+      encode_coding (&coding, src, dst, nbytes, bufsize-2);
+      /* Add the string terminator.  Add two NULs in case we are
+	 producing Unicode here.  */
+      dst[coding.produced] = dst[coding.produced+1] = '\0';
+    }
+
+  if (dst != NULL)
+    GlobalUnlock (htext);
+
+  if (htext != NULL)
+    {
+      /* Shrink data block to actual size.  */
+      htext2 = GlobalReAlloc (htext, coding.produced+2,
+			      GMEM_MOVEABLE | GMEM_DDESHARE);
+      if (htext2 != NULL) htext = htext2;
+    }
+
+  UNGCPRO;
+
+  return htext;
+}
+
+static Lisp_Object
+render (Lisp_Object oformat)
+{
+  HGLOBAL htext = NULL;
+  UINT format = XFASTINT (oformat);
+
+  ONTRACE (fprintf (stderr, "render\n"));
+
+  if (NILP (current_text))
+    return Qnil;
+
+  if (current_requires_encoding || format == CF_UNICODETEXT)
+    {
+      if (format == current_clipboard_type)
+	htext = convert_to_handle_as_coded (current_coding_system);
+      else
+	switch (format)
+	  {
+	  case CF_UNICODETEXT:
+	    htext = convert_to_handle_as_coded (QUNICODE);
+	    break;
+	  case CF_TEXT:
+	  case CF_OEMTEXT:
+	    {
+	      Lisp_Object cs;
+	      cs = coding_from_cp (cp_from_locale (current_lcid, format));
+	      htext = convert_to_handle_as_coded (cs);
+	      break;
+	    }
+	  }
+    }
+  else
+    htext = convert_to_handle_as_ascii ();
+
+  ONTRACE (fprintf (stderr, "render: htext = 0x%08X\n", (unsigned) htext));
+
+  if (htext != NULL)
+    SetClipboardData (format, htext);
+
+  return Qt;
+}
+
+/* At the end of the program, we want to ensure that our clipboard
+   data survives us.  This code will do that.  */
+
+static Lisp_Object
+render_all (void)
+{
+  ONTRACE (fprintf (stderr, "render_all\n"));
 
-  return (ok ? Qt : Qnil);
+  /* According to the docs we should not call OpenClipboard() here,
+     but testing on W2K and working code in other projects shows that
+     it is actually necessary.  */
+
+  OpenClipboard (NULL);
+
+  /* There is no usefull means to report errors here, there are none
+     expected anyway, and even if there were errors, they wouldn't do
+     any harm.  So we just go ahead and do what has to be done without
+     bothering with error handling.  */
+
+  ++modifying_clipboard;
+  EmptyClipboard ();
+  --modifying_clipboard;
+
+  /* For text formats that we don't render here, the OS can use its
+     own translation rules instead, so we don't really need to offer
+     everything.  To minimize memory consumption we cover three
+     possible situations based on our primary format as detected from
+     selection-coding-system (see setup_config()):
+
+     - Post CF_TEXT only.  Let Windows convert to CF_OEMTEXT and
+       Windows on NT or the application on 9x/Me convert to
+       CF_UNICODETEXT.
+
+     - Post CF_OEMTEXT only.  Similar automatic conversions happen as
+       for CF_TEXT.
+
+     - Post CF_UNICODETEXT + CF_TEXT.  On NT/W2K/XP we could even drop
+       the CF_TEXT, because it too could be generated by the OS, but
+       9x ignores CF_UNICODETEXT, even though some applications can
+       still handle it.  */
+
+  render (make_number (current_clipboard_type));
+  if (current_clipboard_type == CF_UNICODETEXT)
+    render (make_number (CF_TEXT));
+
+  CloseClipboard ();
+
+  return Qnil;
 }
 
-DEFUN ("w32-close-clipboard", Fw32_close_clipboard,
-       Sw32_close_clipboard, 0, 0, 0,
-       doc: /* Close the clipboard.  */)
-     ()
+static void
+run_protected (Lisp_Object (*code) (), Lisp_Object arg)
 {
-  BOOL ok = FALSE;
+  extern int waiting_for_input;
+  int owfi;
 
   BLOCK_INPUT;
 
-  ok = CloseClipboard ();
+  /* Fsignal calls abort() if it sees that waiting_for_input set.  */
+  owfi = waiting_for_input;
+  waiting_for_input = 0;
+
+  internal_condition_case_1 (code, arg, Qt, lisp_error_handler);
+
+  waiting_for_input = owfi;
 
   UNBLOCK_INPUT;
+}
 
-  return (ok ? Qt : Qnil);
+static Lisp_Object
+lisp_error_handler (Lisp_Object error)
+{
+  Vsignaling_function = Qnil;
+  cmd_error_internal (error, "Error in delayed clipboard rendering: ");
+  Vinhibit_quit = Qt;
+  return Qt;
 }
 
-#endif
 
-DEFUN ("w32-set-clipboard-data", Fw32_set_clipboard_data,
-       Sw32_set_clipboard_data, 1, 2, 0,
-       doc: /* This sets the clipboard data to the given text.  */)
-    (string, frame)
-    Lisp_Object string, frame;
+static LRESULT CALLBACK
+owner_callback (HWND win, UINT msg, WPARAM wp, LPARAM lp)
 {
-  BOOL ok = TRUE;
-  HANDLE htext;
-  int nbytes;
-  int truelen, nlines = 0;
-  unsigned char *src;
-  unsigned char *dst;
+  switch (msg)
+    {
+    case WM_RENDERFORMAT:
+      ONTRACE (fprintf (stderr, "WM_RENDERFORMAT\n"));
+      run_protected (render, make_number (wp));
+      return 0;
+
+    case WM_RENDERALLFORMATS:
+      ONTRACE (fprintf (stderr, "WM_RENDERALLFORMATS\n"));
+      run_protected (render_all, Qnil);
+      return 0;
 
-  CHECK_STRING (string);
+    case WM_DESTROYCLIPBOARD:
+      if (!modifying_clipboard)
+	{
+	  ONTRACE (fprintf (stderr, "WM_DESTROYCLIPBOARD (other)\n"));
+	  current_text = Qnil;
+	  current_coding_system = Qnil;
+	}
+      else
+	{
+	  ONTRACE (fprintf (stderr, "WM_DESTROYCLIPBOARD (self)\n"));
+	}
+      return 0;
 
-  if (!NILP (frame))
-    CHECK_LIVE_FRAME (frame);
+    case WM_DESTROY:
+      if (win == clipboard_owner)
+	clipboard_owner = NULL;
+      break;
+    }
 
-  BLOCK_INPUT;
+  return DefWindowProc (win, msg, wp, lp);
+}
 
-  /* Include the terminating NULL character in the source of
-     conversion.  */
-  nbytes = SBYTES (string) + 1;
-  src = SDATA (string);
-  dst = src;
+static HWND
+create_owner (void)
+{
+  static const char CLASSNAME[] = "Emacs Clipboard";
+  WNDCLASS wc;
+  HWND hwnd;
+
+  memset (&wc, 0, sizeof (wc));
+  wc.lpszClassName = CLASSNAME;
+  wc.lpfnWndProc = owner_callback;
+  RegisterClass (&wc);
+
+  hwnd = CreateWindow (CLASSNAME, CLASSNAME, 0, 0, 0, 0, 0, NULL, NULL,
+		       NULL, NULL);
+
+  /* FIXME: w32.c says that atexit() is broken.  This works here with
+     Mingw.  The note in w32.c is probably about a static build, which
+     Mingw doesn't support anyway.  Anyway the alternative
+     implementation using signal(SIGABRT) as in w32.c doesn't work
+     here, so standard C is our best bet. */
+
+  if (hwnd != NULL)
+    atexit (atexit_callback);
+
+  return hwnd;
+}
 
-  /* We need to know how many lines there are, since we need CRLF line
-     termination for compatibility with other Windows Programs.
-     avoid using strchr because it recomputes the length every time */
-  while ((dst = memchr (dst, '\n', nbytes - (dst - src))) != NULL)
+static void
+atexit_callback (void)
+{
+  /* Make sure to trigger WM_RENDERALLFORMATS. */
+  if (clipboard_owner != NULL)
+    DestroyWindow (clipboard_owner);
+}
+
+static void
+setup_config (void)
+{
+  const char *coding_name;
+  const char *cp;
+  char *end;
+  int slen;
+  Lisp_Object active_coding_system;
+
+  CHECK_SYMBOL (Vselection_coding_system);
+
+  /* Check if we have it cached */
+  active_coding_system = NILP (Vnext_selection_coding_system) ?
+    Vselection_coding_system : Vnext_selection_coding_system;
+  if (!NILP (cfg_coding_system)
+      && EQ (cfg_coding_system, active_coding_system))
+    return;
+  cfg_coding_system = active_coding_system;
+  
+  /* Set some sensible fallbacks */
+  cfg_codepage = ANSICP;
+  cfg_lcid = LOCALE_NEUTRAL;
+  cfg_clipboard_type = CF_TEXT;
+
+  /* Interpret the coding system symbol name */
+  coding_name = SDATA (SYMBOL_NAME (active_coding_system));
+
+  /* "(.*-)?utf-16.*" -> CF_UNICODETEXT */
+  cp = strstr (coding_name, "utf-16");
+  if (cp != NULL && (cp == coding_name || cp[-1] == '-'))
     {
-      nlines++;
-      dst++;
+      cfg_clipboard_type = CF_UNICODETEXT;
+      return;
     }
 
-  {
-    /* Since we are now handling multilingual text, we must consider
-       encoding text for the clipboard.  */
-    int charset_info = find_charset_in_text (src, SCHARS (string),
-					     nbytes, NULL, Qnil);
+  /* "cp[0-9]+.*" or "windows-[0-9]+.*" -> CF_TEXT or CF_OEMTEXT */
+  slen = strlen (coding_name);
+  if (slen >= 4 && coding_name[0] == 'c' && coding_name[1] == 'p')
+    cp = coding_name + 2;
+  else if (slen >= 10 && memcmp (coding_name, "windows-", 8) == 0)
+    cp = coding_name + 8;
+  else
+    return;
+
+  end = (char*)cp;
+  cfg_codepage = strtol (cp, &end, 10);
+
+  /* Error return from strtol() or number of digits < 2 -> Restore the
+     default and drop it. */
+  if (cfg_codepage == 0 || (end-cp) < 2 )
+    {
+      cfg_codepage = ANSICP;
+      return;
+    }
 
-    if (charset_info == 0)
-      {
-	/* No multibyte character in OBJ.  We need not encode it.  */
+  /* Is it the currently active system default? */
+  if (cfg_codepage == ANSICP)
+    {
+      /* cfg_clipboard_type = CF_TEXT; */
+      return;
+    }
+  if (cfg_codepage == OEMCP)
+    {
+      cfg_clipboard_type = CF_OEMTEXT;
+      return;
+    }
 
-	/* Need to know final size after CR chars are inserted (the
-	   standard CF_TEXT clipboard format uses CRLF line endings,
-	   while Emacs uses just LF internally).  */
+  /* Else determine a suitable locale the hard way. */
+  EnumSystemLocales (enum_locale_callback, LCID_INSTALLED);
+}
 
-	truelen = nbytes + nlines;
+static BOOL WINAPI
+enum_locale_callback (/*const*/ char* loc_string)
+{
+  LCID lcid;
+  UINT codepage;
 
-	if ((htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, truelen)) == NULL)
-	  goto error;
+  lcid = strtoul (loc_string, NULL, 16);
 
-	if ((dst = (unsigned char *) GlobalLock (htext)) == NULL)
-	  goto error;
+  /* Is the wanted codepage the "ANSI" codepage for this locale? */
+  codepage = cp_from_locale (lcid, CF_TEXT);
+  if (codepage == cfg_codepage)
+    {
+      cfg_lcid = lcid;
+      cfg_clipboard_type = CF_TEXT;
+      return FALSE; /* Stop enumeration */
+    }
+  
+  /* Is the wanted codepage the OEM codepage for this locale? */
+  codepage = cp_from_locale (lcid, CF_OEMTEXT);
+  if (codepage == cfg_codepage)
+    {
+      cfg_lcid = lcid;
+      cfg_clipboard_type = CF_OEMTEXT;
+      return FALSE; /* Stop enumeration */
+    }
 
-	/* convert to CRLF line endings expected by clipboard */
-	while (1)
-	  {
-	    unsigned char *next;
-	    /* copy next line or remaining bytes including '\0' */
-	    next = _memccpy (dst, src, '\n', nbytes);
-	    if (next)
-	      {
-		/* copied one line ending with '\n' */
-		int copied = next - dst;
-		nbytes -= copied;
-		src += copied;
-		/* insert '\r' before '\n' */
-		next[-1] = '\r';
-		next[0] = '\n';
-		dst = next + 1;
-	      }
-	    else
-	      /* copied remaining partial line -> now finished */
-	      break;
-	  }
+  return TRUE; /* Continue enumeration */
+}
 
-	GlobalUnlock (htext);
+static UINT
+cp_from_locale (LCID lcid, UINT format)
+{
+  char buffer[20] = "";
+  UINT variant, cp;
 
-	Vlast_coding_system_used = Qraw_text;
-      }
-    else
-      {
-	/* We must encode contents of OBJ to the selection coding
-           system. */
-	int bufsize;
-	struct coding_system coding;
-	HANDLE htext2;
+  variant =
+    format == CF_TEXT ? LOCALE_IDEFAULTANSICODEPAGE : LOCALE_IDEFAULTCODEPAGE;
 
-	if (NILP (Vnext_selection_coding_system))
-	  Vnext_selection_coding_system = Vselection_coding_system;
-	setup_coding_system
-	  (Fcheck_coding_system (Vnext_selection_coding_system), &coding);
-	if (SYMBOLP (coding.pre_write_conversion)
-	    && !NILP (Ffboundp (coding.pre_write_conversion)))
-	  {
-	    string = run_pre_post_conversion_on_str (string, &coding, 1);
-	    src = SDATA (string);
-	    /* Include the terminating NULL character in the source of
-	       conversion.  */
-	    nbytes = SBYTES (string) + 1;
-	  }
-	coding.src_multibyte = 1;
-	coding.dst_multibyte = 0;
-	/* Need to set COMPOSITION_DISABLED, otherwise Emacs crashes in
-	   encode_coding_iso2022 trying to dereference a null pointer.  */
-	coding.composing = COMPOSITION_DISABLED;
-	if (coding.type == coding_type_iso2022)
-	  coding.flags |= CODING_FLAG_ISO_SAFE;
-	Vnext_selection_coding_system = Qnil;
-	coding.mode |= CODING_MODE_LAST_BLOCK;
-	bufsize = encoding_buffer_size (&coding, nbytes);
-	if ((htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, bufsize)) == NULL)
-	  goto error;
-	if ((dst = (unsigned char *) GlobalLock (htext)) == NULL)
-	  goto error;
-	encode_coding (&coding, src, dst, nbytes, bufsize);
-	Vlast_coding_system_used = coding.symbol;
+  GetLocaleInfo (lcid, variant, buffer, sizeof (buffer));
+  cp = strtoul (buffer, NULL, 10);
 
-	/* If clipboard sequence numbers are not supported, keep a copy for
-	   later comparison.  */
-	if (!clipboard_sequence_fn)
-	  {
-	    /* Stash away the data we are about to put into the
-	       clipboard, so we could later check inside
-	       Fw32_get_clipboard_data whether the clipboard still
-	       holds our data.  */
-	    if (clipboard_storage_size < coding.produced)
-	      {
-		clipboard_storage_size = coding.produced + 100;
-		last_clipboard_text = (char *) xrealloc (last_clipboard_text,
-							 clipboard_storage_size);
-	      }
-	    if (last_clipboard_text)
-	      memcpy (last_clipboard_text, dst, coding.produced);
-	  }
+  if (cp == CP_ACP)
+    return ANSICP;
+  else if (cp == CP_OEMCP)
+    return OEMCP;
+  else
+    return cp;
+}
 
-	GlobalUnlock (htext);
+static Lisp_Object
+coding_from_cp (UINT codepage)
+{
+  char buffer[30];
+  sprintf (buffer, "cp%d-dos", (int) codepage);
+  return intern (buffer);
+  /* We don't need to check that this coding system exists right here,
+     because that is done when the coding system is actually
+     instantiated, i.e. it is passed through Fcheck_coding_system()
+     there.  */
+}
 
-	/* Shrink data block to actual size.  */
-	htext2 = GlobalReAlloc (htext, coding.produced,
-                                GMEM_MOVEABLE | GMEM_DDESHARE);
-	if (htext2 != NULL) htext = htext2;
-      }
-  }
 
-  if (!OpenClipboard ((!NILP (frame) && FRAME_W32_P (XFRAME (frame))) ? FRAME_W32_WINDOW (XFRAME (frame)) : NULL))
+DEFUN ("w32-set-clipboard-data", Fw32_set_clipboard_data,
+       Sw32_set_clipboard_data, 1, 2, 0,
+       doc: /* This sets the clipboard data to the given text.  */)
+    (string, ignored)
+    Lisp_Object string, ignored;
+{
+  BOOL ok = TRUE;
+  HGLOBAL hlocale = NULL;
+  int nbytes;
+  unsigned char *src;
+  unsigned char *dst;
+  unsigned char *end;
+
+  (void) ignored;
+
+  CHECK_STRING (string);
+
+  setup_config ();
+
+  current_text = string;
+  current_coding_system = cfg_coding_system;
+  current_clipboard_type = cfg_clipboard_type;
+  current_lcid = cfg_lcid;
+  current_num_nls = 0;
+  current_requires_encoding = 0;
+  
+  BLOCK_INPUT;
+
+  /* Check for non-ASCII characters.  While we are at it, count the
+     number of LFs, so we know how many CRs we will have to add later
+     (just in the case where we can use our internal ASCII rendering,
+     see code and note in convert_to_handle_as_ascii() above).  */
+  nbytes = SBYTES (string);
+  src = SDATA (string);
+
+  for (dst = src, end = src+nbytes; dst < end; dst++)
+    {
+      if (*dst == '\n')
+	current_num_nls++;
+      else if (*dst >= 0x80 || *dst == 0)
+	{
+	  current_requires_encoding = 1;
+	  break;
+	}
+    }
+
+  if (!current_requires_encoding)
+    {
+      /* If all we have is ASCII we don't need to pretend we offer
+	 anything fancy. */
+      current_coding_system = Qraw_text;
+      current_clipboard_type = CF_TEXT;
+    }
+  else
+    {
+      /* If we have something non-ASCII we may want to set a locale.
+	 We always do that directly (non-delayed), as it's just a
+	 small bit and it simplifies the rest of the code. */
+      if (cfg_lcid != LOCALE_NEUTRAL && cfg_lcid != DEFAULT_LCID)
+	{
+	  hlocale = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE,
+				 sizeof (cfg_lcid));
+	  if (hlocale != NULL)
+	    {
+	      LCID * lcid_ptr;
+	      if ((lcid_ptr = (LCID *) GlobalLock (hlocale)) != NULL)
+		{
+		  *lcid_ptr = cfg_lcid;
+		  GlobalUnlock (hlocale);
+		}
+	    }
+	}
+    }
+
+  if (!OpenClipboard (clipboard_owner))
     goto error;
 
-  ok = EmptyClipboard () && SetClipboardData (CF_TEXT, htext);
+  ++modifying_clipboard;
+  ok = EmptyClipboard ();
+  --modifying_clipboard;
+
+  if (ok)
+    {
+      if (clipboard_owner == NULL)
+	{
+	  /* If for some reason we don't have a clipboard_owner, we
+	     just set the text format as chosen by the configuration
+	     and than forget about the whole thing.  */
+	  render (make_number (current_clipboard_type));
+	  current_text = Qnil;
+	  current_coding_system = Qnil;
+	}
+      else
+	{
+	  /* Advertise all supported formats so that whatever the
+	     requester chooses, only one encoding step needs to be
+	     made.  This is intentionally different from what we do in
+	     the WM_RENDERALLFORMATS handler.  */
+	  SetClipboardData (CF_UNICODETEXT, NULL);
+	  SetClipboardData (CF_TEXT, NULL);
+	  SetClipboardData (CF_OEMTEXT, NULL);
+	}
+
+      if (hlocale != NULL)
+	{
+	  ok = SetClipboardData (CF_LOCALE, hlocale) != NULL;
+	  if (!ok) EmptyClipboard ();
+	}
+    }
 
   CloseClipboard ();
 
-  /* Common sense says to read the sequence number inside the
-     OpenClipboard/ CloseClipboard block to avoid race conditions
-     where another app puts something on the clipboard straight after
-     us. But experience suggests that the sequence number from the
-     SetClipboardData is not allocated until we close the clipboard!
-     Since clipboard operations are normally user-driven, the race
-     condition is probably not going to really happen.  */
-  if (clipboard_sequence_fn)
-    last_clipboard_sequence_number = clipboard_sequence_fn ();
+  /* We haven't really "used" this coding system yet, and it's even
+     unclear if we ever will.  But this is a way to tell the upper
+     level what we *would* use under ideal circumstances.
+
+     We don't signal the actually used coding-system later when we
+     finally render, because that can happen at any time and we don't
+     want to disturb the "foreground" action. */
+  Vlast_coding_system_used = current_coding_system;
+
+  Vnext_selection_coding_system = Qnil;
 
   if (ok) goto done;
 
  error:
 
   ok = FALSE;
-  if (htext) GlobalFree (htext);
-  if (last_clipboard_text)
-    *last_clipboard_text = '\0';
+  if (hlocale) GlobalFree (hlocale);
 
-  last_clipboard_sequence_number = 0;
+  current_text = Qnil;
+  current_coding_system = Qnil;
 
  done:
   UNBLOCK_INPUT;
@@ -292,24 +691,50 @@ DEFUN ("w32-set-clipboard-data", Fw32_se
   return (ok ? string : Qnil);
 }
 
+
 DEFUN ("w32-get-clipboard-data", Fw32_get_clipboard_data,
        Sw32_get_clipboard_data, 0, 1, 0,
        doc: /* This gets the clipboard data in text format.  */)
-     (frame)
-     Lisp_Object frame;
+     (ignored)
+     Lisp_Object ignored;
 {
-  HANDLE htext;
+  HGLOBAL htext;
   Lisp_Object ret = Qnil;
+  UINT actual_clipboard_type;
+  int use_configured_coding_system = 1;
+
+  (void) ignored;
 
-  if (!NILP (frame))
-    CHECK_LIVE_FRAME (frame);
+  /* Don't pass our own text from the clipboard (which might be
+     troublesome if the killed text includes null characters).  */
+  if (!NILP (current_text))
+    return ret;
+
+  setup_config ();
+  actual_clipboard_type = cfg_clipboard_type;
 
   BLOCK_INPUT;
 
-  if (!OpenClipboard ((!NILP (frame) && FRAME_W32_P (XFRAME (frame))) ? FRAME_W32_WINDOW (XFRAME (frame)) : NULL))
+  if (!OpenClipboard (clipboard_owner))
     goto done;
 
-  if ((htext = GetClipboardData (CF_TEXT)) == NULL)
+  if ((htext = GetClipboardData (actual_clipboard_type)) == NULL)
+    {
+      /* If we want CF_UNICODETEXT but can't get it, the current
+	 coding system is useless.  OTOH we can still try and decode
+	 CF_TEXT based on the locale that the system gives us and that
+	 we get down below.  */
+      if (actual_clipboard_type == CF_UNICODETEXT)
+	{
+	  htext = GetClipboardData (CF_TEXT);
+	  if (htext != NULL)
+	    {
+	      actual_clipboard_type = CF_TEXT;
+	      use_configured_coding_system = 0;
+	    }
+	}
+    }
+  if (htext == NULL)
     goto closeclip;
 
   {
@@ -322,53 +747,107 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
     if ((src = (unsigned char *) GlobalLock (htext)) == NULL)
       goto closeclip;
 
-    nbytes = strlen (src);
-
-    /* If the text in clipboard is identical to what we put there
-       last time w32_set_clipboard_data was called, pretend there's no
-       data in the clipboard.  This is so we don't pass our own text
-       from the clipboard (which might be troublesome if the killed
-       text includes null characters).  */
-    if ((clipboard_sequence_fn
-	 && clipboard_sequence_fn () == last_clipboard_sequence_number)
-	|| (last_clipboard_text
-	    && clipboard_storage_size >= nbytes
-	    && memcmp(last_clipboard_text, src, nbytes) == 0))
-      goto closeclip;
+    /* If the clipboard data contains any non-ascii code, we need to
+       decode it with a coding system.
+       FIXME: Repeat the code for the Unicode case. */
+    if (actual_clipboard_type == CF_UNICODETEXT)
+      {
+	nbytes = lstrlenW ((WCHAR *)src) * 2;
+	require_decoding = 1;
+      }
+    else
+      {
+	int i;
 
-    {
-      /* If the clipboard data contains any non-ascii code, we
-	 need to decode it.  */
-      int i;
+	nbytes = strlen (src);
 
-      for (i = 0; i < nbytes; i++)
-	{
-	  if (src[i] >= 0x80)
-	    {
-	      require_decoding = 1;
-	      break;
-	    }
-	}
-    }
+	for (i = 0; i < nbytes; i++)
+	  {
+	    if (src[i] >= 0x80)
+	      {
+		require_decoding = 1;
+		break;
+	      }
+	  }
+      }
 
     if (require_decoding)
       {
 	int bufsize;
 	unsigned char *buf;
 	struct coding_system coding;
+	Lisp_Object coding_system = Qnil;
+	
+	/* `next-selection-coding-system' should override everything,
+	   even when the locale passed by the system disagrees.  The
+	   only exception is when `next-selection-coding-system'
+	   requested CF_UNICODETEXT and we couldn't get that. */
+	if (use_configured_coding_system
+	    && !NILP (Vnext_selection_coding_system))
+	    coding_system = Vnext_selection_coding_system;
+
+	/* If we have CF_TEXT or CF_OEMTEXT, we want to check out
+	   CF_LOCALE, too. */
+	else if (actual_clipboard_type != CF_UNICODETEXT)
+	  {
+	    HGLOBAL hlocale;
+	    LCID lcid = DEFAULT_LCID;
+	    UINT cp;
+
+	    /* Windows is documented to always generate CF_LOCALE info
+	       automatically, so the locale handle should be always
+	       present, but this is not always true on 9x ;-(.  */
+	    hlocale = GetClipboardData (CF_LOCALE);
+	    if (hlocale != NULL)
+	      {
+		const LCID * lcid_ptr;
+		lcid_ptr = (const LCID *) GlobalLock (hlocale);
+		if (lcid_ptr != NULL)
+		  {
+		    lcid = *lcid_ptr;
+		    GlobalUnlock (hlocale);
+		  }
+
+		/* 9x has garbage as the sort order (to be exact there
+		   is another instance of the language id in the upper
+		   word).  We don't care about sort order anyway, so
+		   we can just filter out the unneeded mis-information
+		   to avoid irritations. */
+		lcid = MAKELCID (LANGIDFROMLCID (lcid), SORT_DEFAULT);
+	      }
 
-	if (NILP (Vnext_selection_coding_system))
-	  Vnext_selection_coding_system = Vselection_coding_system;
-	setup_coding_system
-	  (Fcheck_coding_system (Vnext_selection_coding_system), &coding);
+	    /* If we are using fallback from CF_UNICODETEXT, we can't
+	       use the configured coding system.  Also we don't want
+	       to use it, if the system has supplied us with a locale
+	       and it is not just the system default. */
+	    if (!use_configured_coding_system || lcid != DEFAULT_LCID)
+	      {
+		cp = cp_from_locale (lcid, actual_clipboard_type);
+		/* If it's just our current standard setting anyway,
+		   use the coding system that the user has selected.
+		   Otherwise create a new spec to match the locale
+		   that was specified by the other side or the
+		   system.  */
+		if (!use_configured_coding_system || cp != cfg_codepage)
+		  coding_system = coding_from_cp (cp);
+	      }
+	  }
+
+	if (NILP (coding_system))
+	  coding_system = Vselection_coding_system;
+	Vnext_selection_coding_system = Qnil;
+
+	setup_coding_system (Fcheck_coding_system (coding_system), &coding);
 	coding.src_multibyte = 0;
 	coding.dst_multibyte = 1;
-	Vnext_selection_coding_system = Qnil;
 	coding.mode |= CODING_MODE_LAST_BLOCK;
 	/* We explicitely disable composition handling because
 	   selection data should not contain any composition
 	   sequence.  */
 	coding.composing = COMPOSITION_DISABLED;
+	/* Force DOS line-ends. */
+	coding.eol_type = CODING_EOL_CRLF;
+
 	bufsize = decoding_buffer_size (&coding, nbytes);
 	buf = (unsigned char *) xmalloc (bufsize);
 	decode_coding (&coding, src, buf, nbytes, bufsize);
@@ -382,10 +861,10 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
       }
     else
       {
-	/* Need to know final size after CR chars are removed because we
-	   can't change the string size manually, and doing an extra
-	   copy is silly.  Note that we only remove CR when it appears
-	   as part of CRLF.  */
+	/* Need to know final size after CR chars are removed because
+	   we can't change the string size manually, and doing an
+	   extra copy is silly.  We only remove CR when it appears as
+	   part of CRLF.  */
 
 	truelen = nbytes;
 	dst = src;
@@ -462,9 +941,14 @@ and t is the same as `SECONDARY'.  */)
 
       if (OpenClipboard (NULL))
 	{
-	  int format = 0;
-	  while (format = EnumClipboardFormats (format))
-	    if (format == CF_TEXT)
+	  UINT format = 0;
+	  setup_config ();
+	  while ((format = EnumClipboardFormats (format)))
+	    /* Check CF_TEXT in addition to cfg_clipboard_type,
+	       because we can fall back on that if CF_UNICODETEXT is
+	       not available.  Actually a check for CF_TEXT only
+	       should be enough.  */
+	    if (format == cfg_clipboard_type || format == CF_TEXT)
 	      {
 		val = Qt;
 		break;
@@ -476,24 +960,25 @@ and t is the same as `SECONDARY'.  */)
   return Qnil;
 }
 
+/* One-time init.  Called in the un-dumped Emacs, but not in the
+   dumped version. */
+
 void
 syms_of_w32select ()
 {
-#if 0
-  defsubr (&Sw32_open_clipboard);
-  defsubr (&Sw32_empty_clipboard);
-  defsubr (&Sw32_close_clipboard);
-#endif
   defsubr (&Sw32_set_clipboard_data);
   defsubr (&Sw32_get_clipboard_data);
   defsubr (&Sx_selection_exists_p);
 
   DEFVAR_LISP ("selection-coding-system", &Vselection_coding_system,
 	       doc: /* Coding system for communicating with other programs.
-When sending or receiving text via cut_buffer, selection, and clipboard,
-the text is encoded or decoded by this coding system.
-The default value is `iso-latin-1-dos'.  */);
-  Vselection_coding_system = intern ("iso-latin-1-dos");
+When sending or receiving text via cut_buffer, selection, and
+clipboard, the text is encoded or decoded by this coding system.
+The default value is the current system default encoding on 9x/Me and
+`utf-16le-dos' (Unicode) on NT/W2K/XP. */);  
+  /* The actual value is set dynamically in the dumped Emacs, see
+     below. */
+  Vselection_coding_system = Qnil;
 
   DEFVAR_LISP ("next-selection-coding-system", &Vnext_selection_coding_system,
 	       doc: /* Coding system for the next communication with other programs.
@@ -504,6 +989,41 @@ set to nil.  */);
   Vnext_selection_coding_system = Qnil;
 
   QCLIPBOARD = intern ("CLIPBOARD");	staticpro (&QCLIPBOARD);
+
+  cfg_coding_system = Qnil;     staticpro (&cfg_coding_system);
+  current_text = Qnil;		staticpro (&current_text);
+  current_coding_system = Qnil; staticpro (&current_coding_system);
+
+  QUNICODE = intern ("utf-16le-dos"); staticpro (&QUNICODE);
+  QANSICP = Qnil; staticpro (&QANSICP);
+  QOEMCP = Qnil;  staticpro (&QOEMCP);
+}
+
+/* One-time init.  Called in the dumped Emacs, but not in the
+   un-dumped version. */
+
+void
+globals_of_w32select ()
+{
+  DEFAULT_LCID = GetUserDefaultLCID ();
+  /* Drop the sort order from the LCID, so we can compare this with
+     CF_LOCALE objects that have the same fix on 9x.  */
+  DEFAULT_LCID = MAKELCID (LANGIDFROMLCID (DEFAULT_LCID), SORT_DEFAULT);
+
+  ANSICP = GetACP ();
+  OEMCP = GetOEMCP ();
+
+  QANSICP = coding_from_cp (ANSICP);
+  QOEMCP = coding_from_cp (OEMCP);
+
+  if (os_subtype == OS_NT)
+    Vselection_coding_system = QUNICODE;
+  else if (inhibit_window_system)
+    Vselection_coding_system = QOEMCP;
+  else
+    Vselection_coding_system = QANSICP;
+
+  clipboard_owner = create_owner ();
 }
 
 /* arch-tag: c96e9724-5eb1-4dad-be07-289f092fd2af

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Unicode support for the MS Windows clipboard

[Jason Rumney]

Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:

>> Somebody suggested to automatically map all Unicode coding-systems
>> to the right one, utf-16le-dos.
>
> The uncertainty about the right Unicode coding-system to use seems to
> me to be a general problem with the naming of the Unicode
> coding-systems, not with w32select.c.

My point was that users shouldn't have to know the differences
between little-endian and big-endian and with/without signatures just
to use the Windows clipboard. No matter what we call those coding
systems, users will still need intimate knowledge of Unicode to know
what they mean.

Re: Unicode support for the MS Windows clipboard

[Jason Rumney]

Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:

>> I get the feeling that the C code does too much, and that it should
>> delegate major parts of the processing to Lisp.  I am not yet sure
>> how to structure that, though.
>
> After thinking about it, making more interfaces between Lisp and C
> doesn't seem too usefull.  Windows is pretty firm about what is what
> here, so there is not much to customize.  Via the
> selection-coding-system variable, the algorithm is already more
> tinkerable from the Lisp side than strictly necessary.

It would be useful do the clipboard rendering from Lisp, like the
clipboard support for X does. This could enable other formats (rich
text, images) to be supported without C code changes. This was
actually one of the main reasons for implementing delayed rendering.

Re: Unicode support for the MS Windows clipboard

[Eli Zaretskii]

> From: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de>
> Date: Mon, 26 Jul 2004 21:17:13 +0200
> 
> Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:
> > While I am still familiar with the internal of this module, I'd like
> > to tackle delayed rendering.
> 
> Done that, see attached expanded patch. 

Thanks.

I skimmed through the code, and it seems okay to me.  But please wait
for Jason to make more specific comments if he has them.

One thing that I think is missing from these changes is a suitable
change to the user manual, where the Windows clipboard encoding is
described.  A NEWS entry is also in order, I think.

Re: Unicode support for the MS Windows clipboard

[Jason Rumney]

The only comment I have on the actual code is about the following

+  render (make_number (current_clipboard_type));
+  if (current_clipboard_type == CF_UNICODETEXT)
+    render (make_number (CF_TEXT));

Make sure that rendering CF_TEXT does not clobber the CF_UNICODETEXT
that you just put there on the NT based versions of Windows that do
automatic conversions. It might be better to reverse the order.

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

Hi Jason,

Jason Rumney <jasonr@gnu.org> writes:
> +  render (make_number (current_clipboard_type));
> +  if (current_clipboard_type == CF_UNICODETEXT)
> +    render (make_number (CF_TEXT));
>
> Make sure that rendering CF_TEXT does not clobber the CF_UNICODETEXT
> that you just put there on the NT based versions of Windows that do
> automatic conversions. It might be better to reverse the order.

*Very* good point, thanks. 

benny

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

Hi Jason,


Jason Rumney <jasonr@gnu.org> writes:
> It would be useful do the clipboard rendering from Lisp, like the
> clipboard support for X does. This could enable other formats (rich
> text, images) to be supported without C code changes.

It should not be a problem to add a callback interface to do the
rendering/encoding, that would be just a further extension of the code
as I posted it.  It's not something that I am currently personally
interested in doing though.

(Note that a more direct mapping of Windows facilities (like exposing
the clipboard formats) was in my very first experimental patch.  At
the time somebody said, that this wasn't good, you didn't want to
support more on Windows than you support on X11.  I may have
misunderstood something here.)


benny

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

Hi Eli,

"Eli Zaretskii" <eliz@gnu.org> writes:
> One thing that I think is missing from these changes is a suitable
> change to the user manual, where the Windows clipboard encoding is
> described.  A NEWS entry is also in order, I think.

I'll try to think of something. 

benny

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

[-- Attachment #1: Type: text/plain, Size: 479 bytes --]

Hi Jason,

Jason Rumney <jasonr@gnu.org> writes:
> Make sure that rendering CF_TEXT does not clobber the CF_UNICODETEXT
> that you just put there on the NT based versions of Windows that do
> automatic conversions. It might be better to reverse the order.

It's not a problem on W2K here and thinking about it, it would be a
bug in the OS if it was, automatic conversion is supposed to be a
fallback solution.  But better be safe than sorry.  See attached
revised patch.

benny


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: w32select.patch --]
[-- Type: text/x-patch, Size: 40030 bytes --]

Index: lisp/international/mule-cmds.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/international/mule-cmds.el,v
retrieving revision 1.257
diff -u -p -r1.257 mule-cmds.el
--- lisp/international/mule-cmds.el	12 Jun 2004 02:21:21 -0000	1.257
+++ lisp/international/mule-cmds.el	27 Jul 2004 12:10:46 -0000
@@ -2422,13 +2422,13 @@ See also `locale-charset-language-names'
 	  (prefer-coding-system coding-system)
 	  (setq locale-coding-system coding-system))))
 
-    ;; On Windows, override locale-coding-system, keyboard-coding-system,
-    ;; selection-coding-system with system codepage.
+    ;; On Windows, override locale-coding-system,
+    ;; keyboard-coding-system, with system codepage.  Note:
+    ;; selection-coding-system is already set in w32select.c.
     (when (boundp 'w32-ansi-code-page)
       (let ((code-page-coding (intern (format "cp%d" w32-ansi-code-page))))
 	(when (coding-system-p code-page-coding)
 	  (setq locale-coding-system code-page-coding)
-	  (set-selection-coding-system code-page-coding)
 	  (set-keyboard-coding-system code-page-coding)
 	  (set-terminal-coding-system code-page-coding))))
 
Index: src/emacs.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/emacs.c,v
retrieving revision 1.342
diff -u -p -r1.342 emacs.c
--- src/emacs.c	24 Jun 2004 20:24:52 -0000	1.342
+++ src/emacs.c	27 Jul 2004 12:11:01 -0000
@@ -42,6 +42,8 @@ Boston, MA 02111-1307, USA.  */
 
 #ifdef WINDOWSNT
 #include <fcntl.h>
+#include <windows.h> /* just for w32.h */
+#include "w32.h"
 #endif
 
 #include "lisp.h"
@@ -1578,6 +1580,7 @@ main (argc, argv
 #ifdef HAVE_NTGUI
       globals_of_w32fns ();
       globals_of_w32menu ();
+      globals_of_w32select ();
 #endif  /* end #ifdef HAVE_NTGUI */
     }
 
Index: src/w32.h
===================================================================
RCS file: /cvsroot/emacs/emacs/src/w32.h,v
retrieving revision 1.15
diff -u -p -r1.15 w32.h
--- src/w32.h	1 Sep 2003 15:45:57 -0000	1.15
+++ src/w32.h	27 Jul 2004 12:11:10 -0000
@@ -122,16 +122,17 @@ extern void reset_standard_handles (int 
 /* Return the string resource associated with KEY of type TYPE.  */
 extern LPBYTE w32_get_resource (char * key, LPDWORD type);
 
-extern void init_ntproc ();
-extern void term_ntproc ();
-extern void globals_of_w32 ();
-extern void syms_of_w32term ();
-extern void syms_of_w32fns ();
-extern void globals_of_w32fns ();
-extern void syms_of_w32select ();
-extern void syms_of_w32menu ();
-extern void globals_of_w32menu ();
-extern void syms_of_fontset ();
+extern void init_ntproc (void);
+extern void term_ntproc (void);
+extern void globals_of_w32 (void);
+extern void syms_of_w32term (void);
+extern void syms_of_w32fns (void);
+extern void globals_of_w32fns (void);
+extern void syms_of_w32select (void);
+extern void globals_of_w32select (void);
+extern void syms_of_w32menu (void);
+extern void globals_of_w32menu (void);
+extern void syms_of_fontset (void);
 
 #endif /* EMACS_W32_H */
 
Index: src/w32select.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/w32select.c,v
retrieving revision 1.33
diff -u -p -r1.33 w32select.c
--- src/w32select.c	30 Jun 2004 23:30:39 -0000	1.33
+++ src/w32select.c	27 Jul 2004 12:11:10 -0000
@@ -1,5 +1,5 @@
 /* Selection processing for Emacs on the Microsoft W32 API.
-   Copyright (C) 1993, 1994 Free Software Foundation.
+   Copyright (C) 1993, 1994, 2004 Free Software Foundation.
 
 This file is part of GNU Emacs.
 
@@ -18,273 +18,681 @@ along with GNU Emacs; see the file COPYI
 the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
 Boston, MA 02111-1307, USA.  */
 
-/* Written by Kevin Gallo */
+/* Written by Kevin Gallo, Benjamin Riefenstahl */
 
 #include <config.h>
 #include "lisp.h"
 #include "w32term.h"	/* for all of the w32 includes */
-#include "dispextern.h"	/* frame.h seems to want this */
-#include "keyboard.h"
-#include "frame.h"	/* Need this to get the X window of selected_frame */
+#include "w32heap.h"	/* os_subtype */
 #include "blockinput.h"
-#include "buffer.h"
+#include "keyboard.h"	/* cmd_error_internal() */
 #include "charset.h"
 #include "coding.h"
 #include "composite.h"
 
+
+static HGLOBAL convert_to_handle_as_ascii (void);
+static HGLOBAL convert_to_handle_as_coded (Lisp_Object coding_system);
+static Lisp_Object render (Lisp_Object oformat);
+static Lisp_Object render_all (void);
+static void run_protected (Lisp_Object (*code) (), Lisp_Object arg);
+static Lisp_Object lisp_error_handler (Lisp_Object error);
+static LRESULT CALLBACK owner_callback (HWND win, UINT msg,
+					WPARAM wp, LPARAM lp);
+static HWND create_owner (void);
+static void atexit_callback (void);
+
+static void setup_config (void);
+static BOOL WINAPI enum_locale_callback (/*const*/ char* loc_string);
+static UINT cp_from_locale (LCID lcid, UINT format);
+static Lisp_Object coding_from_cp (UINT codepage);
+
+
 Lisp_Object QCLIPBOARD;
 
 /* Coding system for communicating with other Windows programs via the
    clipboard.  */
 static Lisp_Object Vselection_coding_system;
 
-/* Coding system for the next communicating with other Windows programs.  */
+/* Coding system for the next communication with other Windows
+   programs.  */
 static Lisp_Object Vnext_selection_coding_system;
 
-/* Sequence number, used where possible to detect when we are pasting
-   our own text.  */
-static DWORD last_clipboard_sequence_number;
-extern ClipboardSequence_Proc clipboard_sequence_fn;
-
-/* The last text we put into the clipboard.  This is used when the OS
-   does not support sequence numbers (NT4, 95). It is undesirable to
-   use data put on the clipboard by Emacs because the clipboard data
-   could be MULEtilated by inappropriately chosen
-   (next-)selection-coding-system.  For this reason, we must store the
-   text *after* it was encoded/Unix-to-DOS-converted.  */
-static unsigned char *last_clipboard_text = NULL;
-static size_t clipboard_storage_size = 0;
-
-#if 0
-DEFUN ("w32-open-clipboard", Fw32_open_clipboard, Sw32_open_clipboard, 0, 1, 0,
-       doc: /* This opens the clipboard with the given frame pointer.  */)
-     (frame)
-     Lisp_Object frame;
+/* Internal pseudo-constants, initialized in globals_of_w32select()
+   based on current system parameters. */
+static LCID DEFAULT_LCID;
+static UINT ANSICP, OEMCP;
+static Lisp_Object QUNICODE, QANSICP, QOEMCP;
+
+/* A hidden window just for the clipboard management. */
+static HWND clipboard_owner;
+/* A flag to tell WM_DESTROYCLIPBOARD who is to blame this time (just
+   checking GetClipboardOwner() doesn't work, sadly). */
+static int modifying_clipboard = 0;
+
+/* Configured transfer parameters, based on the last inspection of
+   selection-coding-system.  */
+static Lisp_Object cfg_coding_system;
+static UINT cfg_codepage;
+static LCID cfg_lcid;
+static UINT cfg_clipboard_type;
+
+/* The current state for delayed rendering. */
+static Lisp_Object current_text;
+static Lisp_Object current_coding_system;
+static int current_requires_encoding, current_num_nls;
+static UINT current_clipboard_type;
+static LCID current_lcid;
+
+#if TRACE
+#define ONTRACE(stmt) stmt
+#else
+#define ONTRACE(stmt) /*stmt*/
+#endif
+
+
+/* This function assumes that there is no multibyte character in
+   current_text, so we can short-cut encoding.  */
+
+static HGLOBAL
+convert_to_handle_as_ascii (void)
 {
-  BOOL ok = FALSE;
+  HGLOBAL htext = NULL;
+  int nbytes;
+  int truelen;
+  unsigned char *src;
+  unsigned char *dst;
 
-  if (!NILP (frame))
-    CHECK_LIVE_FRAME (frame);
+  ONTRACE (fprintf (stderr, "convert_to_handle_as_ascii\n"));
 
-  BLOCK_INPUT;
+  nbytes = SBYTES (current_text) + 1;
+  src = SDATA (current_text);
 
-  ok = OpenClipboard ((!NILP (frame) && FRAME_W32_P (XFRAME (frame))) ? FRAME_W32_WINDOW (XFRAME (frame)) : NULL);
+  /* We need to add to the size the number of LF chars where we have
+     to insert CR chars (the standard CF_TEXT clipboard format uses
+     CRLF line endings, while Emacs uses just LF internally).  */
 
-  UNBLOCK_INPUT;
+  truelen = nbytes + current_num_nls;
+
+  if ((htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, truelen)) == NULL)
+    return NULL;
+
+  if ((dst = (unsigned char *) GlobalLock (htext)) == NULL)
+    {
+      GlobalFree (htext);
+      return NULL;
+    }
+
+  /* convert to CRLF line endings expected by clipboard */
+  while (1)
+    {
+      unsigned char *next;
+      /* copy next line or remaining bytes including '\0' */
+      next = _memccpy (dst, src, '\n', nbytes);
+      if (next)
+	{
+	  /* copied one line ending with '\n' */
+	  int copied = next - dst;
+	  nbytes -= copied;
+	  src += copied;
+	  /* insert '\r' before '\n' */
+	  next[-1] = '\r';
+	  next[0] = '\n';
+	  dst = next + 1;
+	}
+      else
+	/* copied remaining partial line -> now finished */
+	break;
+    }
 
-  return (ok ? frame : Qnil);
+  GlobalUnlock (htext);
+
+  return htext;
 }
 
-DEFUN ("w32-empty-clipboard", Fw32_empty_clipboard,
-       Sw32_empty_clipboard, 0, 0, 0,
-       doc: /* Empty the clipboard.
-Assigns ownership of the clipboard to the window which opened it.  */)
-     ()
+/* This function assumes that there are multibyte or NUL characters in
+   current_text, or that we need to construct Unicode.  It runs the
+   text through the encoding machinery.  */
+
+static HGLOBAL
+convert_to_handle_as_coded (Lisp_Object coding_system)
 {
-  BOOL ok = FALSE;
+  HGLOBAL htext = NULL, htext2;
+  int nbytes;
+  unsigned char *src;
+  unsigned char *dst = NULL;
+  int bufsize;
+  struct coding_system coding;
+  Lisp_Object string = Qnil;
+  struct gcpro gcpro1;
+
+  ONTRACE (fprintf (stderr, "convert_to_handle_as_coded: %s\n",	
+		    SDATA (SYMBOL_NAME (coding_system))));
+
+  /* FIXME: It looks like GC_MARK_STACK == GC_MAKE_GCPROS_NOOPS on
+     Windows (see src/lisp.h and nt/config.nt), so this is probably
+     unnecessary.  If it was necessary, we'd probably have to protect
+     all the symbols, too. */
+  GCPRO1 (string);
+
+  setup_coding_system (Fcheck_coding_system (coding_system), &coding);
+  coding.src_multibyte = 1;
+  coding.dst_multibyte = 0;
+  /* Need to set COMPOSITION_DISABLED, otherwise Emacs crashes in
+     encode_coding_iso2022 trying to dereference a null pointer.  */
+  coding.composing = COMPOSITION_DISABLED;
+  if (coding.type == coding_type_iso2022)
+    coding.flags |= CODING_FLAG_ISO_SAFE;
+  coding.mode |= CODING_MODE_LAST_BLOCK;
+  /* Force DOS line-ends. */
+  coding.eol_type = CODING_EOL_CRLF;
+
+  if (SYMBOLP (coding.pre_write_conversion)
+      && !NILP (Ffboundp (coding.pre_write_conversion)))
+    string = run_pre_post_conversion_on_str (current_text, &coding, 1);
+  else
+    string = current_text;
 
-  BLOCK_INPUT;
+  nbytes = SBYTES (string);
+  src = SDATA (string);
 
-  ok = EmptyClipboard ();
+  bufsize = encoding_buffer_size (&coding, nbytes) +2;
+  htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, bufsize);
 
-  UNBLOCK_INPUT;
+  if (htext != NULL)
+    dst = (unsigned char *) GlobalLock (htext);
+
+  if (dst != NULL)
+    {
+      encode_coding (&coding, src, dst, nbytes, bufsize-2);
+      /* Add the string terminator.  Add two NULs in case we are
+	 producing Unicode here.  */
+      dst[coding.produced] = dst[coding.produced+1] = '\0';
+    }
+
+  if (dst != NULL)
+    GlobalUnlock (htext);
+
+  if (htext != NULL)
+    {
+      /* Shrink data block to actual size.  */
+      htext2 = GlobalReAlloc (htext, coding.produced+2,
+			      GMEM_MOVEABLE | GMEM_DDESHARE);
+      if (htext2 != NULL) htext = htext2;
+    }
+
+  UNGCPRO;
+
+  return htext;
+}
+
+static Lisp_Object
+render (Lisp_Object oformat)
+{
+  HGLOBAL htext = NULL;
+  UINT format = XFASTINT (oformat);
+
+  ONTRACE (fprintf (stderr, "render\n"));
+
+  if (NILP (current_text))
+    return Qnil;
+
+  if (current_requires_encoding || format == CF_UNICODETEXT)
+    {
+      if (format == current_clipboard_type)
+	htext = convert_to_handle_as_coded (current_coding_system);
+      else
+	switch (format)
+	  {
+	  case CF_UNICODETEXT:
+	    htext = convert_to_handle_as_coded (QUNICODE);
+	    break;
+	  case CF_TEXT:
+	  case CF_OEMTEXT:
+	    {
+	      Lisp_Object cs;
+	      cs = coding_from_cp (cp_from_locale (current_lcid, format));
+	      htext = convert_to_handle_as_coded (cs);
+	      break;
+	    }
+	  }
+    }
+  else
+    htext = convert_to_handle_as_ascii ();
+
+  ONTRACE (fprintf (stderr, "render: htext = 0x%08X\n", (unsigned) htext));
+
+  if (htext != NULL)
+    SetClipboardData (format, htext);
+
+  return Qt;
+}
+
+/* At the end of the program, we want to ensure that our clipboard
+   data survives us.  This code will do that.  */
+
+static Lisp_Object
+render_all (void)
+{
+  ONTRACE (fprintf (stderr, "render_all\n"));
 
-  return (ok ? Qt : Qnil);
+  /* According to the docs we should not call OpenClipboard() here,
+     but testing on W2K and working code in other projects shows that
+     it is actually necessary.  */
+
+  OpenClipboard (NULL);
+
+  /* There is no usefull means to report errors here, there are none
+     expected anyway, and even if there were errors, they wouldn't do
+     any harm.  So we just go ahead and do what has to be done without
+     bothering with error handling.  */
+
+  ++modifying_clipboard;
+  EmptyClipboard ();
+  --modifying_clipboard;
+
+  /* For text formats that we don't render here, the OS can use its
+     own translation rules instead, so we don't really need to offer
+     everything.  To minimize memory consumption we cover three
+     possible situations based on our primary format as detected from
+     selection-coding-system (see setup_config()):
+
+     - Post CF_TEXT only.  Let Windows convert to CF_OEMTEXT and
+       Windows on NT or the application on 9x/Me convert to
+       CF_UNICODETEXT.
+
+     - Post CF_OEMTEXT only.  Similar automatic conversions happen as
+       for CF_TEXT.
+
+     - Post CF_UNICODETEXT + CF_TEXT.  9x itself ignores
+       CF_UNICODETEXT, even though some applications can still handle
+       it.
+
+       Note 1: We render the less capable CF_TEXT *before* the more
+       capable CF_UNICODETEXT, to prevent clobbering through automatic
+       conversions, just in case.
+
+       Note 2: We could check os_subtype here and only render the
+       additional CF_TEXT on 9x/Me.  But OTOH with
+       current_clipboard_type == CF_UNICODETEXT we don't involve the
+       automatic conversions anywhere else, so to get consistent
+       results, we probably don't want to rely on it here either.  */
+
+  if (current_clipboard_type == CF_UNICODETEXT)
+    render (make_number (CF_TEXT));
+  render (make_number (current_clipboard_type));
+
+  CloseClipboard ();
+
+  return Qnil;
 }
 
-DEFUN ("w32-close-clipboard", Fw32_close_clipboard,
-       Sw32_close_clipboard, 0, 0, 0,
-       doc: /* Close the clipboard.  */)
-     ()
+static void
+run_protected (Lisp_Object (*code) (), Lisp_Object arg)
 {
-  BOOL ok = FALSE;
+  extern int waiting_for_input;
+  int owfi;
 
   BLOCK_INPUT;
 
-  ok = CloseClipboard ();
+  /* Fsignal calls abort() if it sees that waiting_for_input set.  */
+  owfi = waiting_for_input;
+  waiting_for_input = 0;
+
+  internal_condition_case_1 (code, arg, Qt, lisp_error_handler);
+
+  waiting_for_input = owfi;
 
   UNBLOCK_INPUT;
+}
 
-  return (ok ? Qt : Qnil);
+static Lisp_Object
+lisp_error_handler (Lisp_Object error)
+{
+  Vsignaling_function = Qnil;
+  cmd_error_internal (error, "Error in delayed clipboard rendering: ");
+  Vinhibit_quit = Qt;
+  return Qt;
 }
 
-#endif
 
-DEFUN ("w32-set-clipboard-data", Fw32_set_clipboard_data,
-       Sw32_set_clipboard_data, 1, 2, 0,
-       doc: /* This sets the clipboard data to the given text.  */)
-    (string, frame)
-    Lisp_Object string, frame;
+static LRESULT CALLBACK
+owner_callback (HWND win, UINT msg, WPARAM wp, LPARAM lp)
 {
-  BOOL ok = TRUE;
-  HANDLE htext;
-  int nbytes;
-  int truelen, nlines = 0;
-  unsigned char *src;
-  unsigned char *dst;
+  switch (msg)
+    {
+    case WM_RENDERFORMAT:
+      ONTRACE (fprintf (stderr, "WM_RENDERFORMAT\n"));
+      run_protected (render, make_number (wp));
+      return 0;
+
+    case WM_RENDERALLFORMATS:
+      ONTRACE (fprintf (stderr, "WM_RENDERALLFORMATS\n"));
+      run_protected (render_all, Qnil);
+      return 0;
 
-  CHECK_STRING (string);
+    case WM_DESTROYCLIPBOARD:
+      if (!modifying_clipboard)
+	{
+	  ONTRACE (fprintf (stderr, "WM_DESTROYCLIPBOARD (other)\n"));
+	  current_text = Qnil;
+	  current_coding_system = Qnil;
+	}
+      else
+	{
+	  ONTRACE (fprintf (stderr, "WM_DESTROYCLIPBOARD (self)\n"));
+	}
+      return 0;
 
-  if (!NILP (frame))
-    CHECK_LIVE_FRAME (frame);
+    case WM_DESTROY:
+      if (win == clipboard_owner)
+	clipboard_owner = NULL;
+      break;
+    }
 
-  BLOCK_INPUT;
+  return DefWindowProc (win, msg, wp, lp);
+}
 
-  /* Include the terminating NULL character in the source of
-     conversion.  */
-  nbytes = SBYTES (string) + 1;
-  src = SDATA (string);
-  dst = src;
+static HWND
+create_owner (void)
+{
+  static const char CLASSNAME[] = "Emacs Clipboard";
+  WNDCLASS wc;
+  HWND hwnd;
+
+  memset (&wc, 0, sizeof (wc));
+  wc.lpszClassName = CLASSNAME;
+  wc.lpfnWndProc = owner_callback;
+  RegisterClass (&wc);
+
+  hwnd = CreateWindow (CLASSNAME, CLASSNAME, 0, 0, 0, 0, 0, NULL, NULL,
+		       NULL, NULL);
+
+  /* FIXME: w32.c says that atexit() is broken.  This works here with
+     Mingw.  The note in w32.c is probably about a static build, which
+     Mingw doesn't support anyway.  Anyway the alternative
+     implementation using signal(SIGABRT) as in w32.c doesn't work
+     here, so standard C is our best bet. */
+
+  if (hwnd != NULL)
+    atexit (atexit_callback);
+
+  return hwnd;
+}
 
-  /* We need to know how many lines there are, since we need CRLF line
-     termination for compatibility with other Windows Programs.
-     avoid using strchr because it recomputes the length every time */
-  while ((dst = memchr (dst, '\n', nbytes - (dst - src))) != NULL)
+static void
+atexit_callback (void)
+{
+  /* Make sure to trigger WM_RENDERALLFORMATS. */
+  if (clipboard_owner != NULL)
+    DestroyWindow (clipboard_owner);
+}
+
+static void
+setup_config (void)
+{
+  const char *coding_name;
+  const char *cp;
+  char *end;
+  int slen;
+  Lisp_Object active_coding_system;
+
+  CHECK_SYMBOL (Vselection_coding_system);
+
+  /* Check if we have it cached */
+  active_coding_system = NILP (Vnext_selection_coding_system) ?
+    Vselection_coding_system : Vnext_selection_coding_system;
+  if (!NILP (cfg_coding_system)
+      && EQ (cfg_coding_system, active_coding_system))
+    return;
+  cfg_coding_system = active_coding_system;
+  
+  /* Set some sensible fallbacks */
+  cfg_codepage = ANSICP;
+  cfg_lcid = LOCALE_NEUTRAL;
+  cfg_clipboard_type = CF_TEXT;
+
+  /* Interpret the coding system symbol name */
+  coding_name = SDATA (SYMBOL_NAME (active_coding_system));
+
+  /* "(.*-)?utf-16.*" -> CF_UNICODETEXT */
+  cp = strstr (coding_name, "utf-16");
+  if (cp != NULL && (cp == coding_name || cp[-1] == '-'))
     {
-      nlines++;
-      dst++;
+      cfg_clipboard_type = CF_UNICODETEXT;
+      return;
     }
 
-  {
-    /* Since we are now handling multilingual text, we must consider
-       encoding text for the clipboard.  */
-    int charset_info = find_charset_in_text (src, SCHARS (string),
-					     nbytes, NULL, Qnil);
+  /* "cp[0-9]+.*" or "windows-[0-9]+.*" -> CF_TEXT or CF_OEMTEXT */
+  slen = strlen (coding_name);
+  if (slen >= 4 && coding_name[0] == 'c' && coding_name[1] == 'p')
+    cp = coding_name + 2;
+  else if (slen >= 10 && memcmp (coding_name, "windows-", 8) == 0)
+    cp = coding_name + 8;
+  else
+    return;
+
+  end = (char*)cp;
+  cfg_codepage = strtol (cp, &end, 10);
+
+  /* Error return from strtol() or number of digits < 2 -> Restore the
+     default and drop it. */
+  if (cfg_codepage == 0 || (end-cp) < 2 )
+    {
+      cfg_codepage = ANSICP;
+      return;
+    }
 
-    if (charset_info == 0)
-      {
-	/* No multibyte character in OBJ.  We need not encode it.  */
+  /* Is it the currently active system default? */
+  if (cfg_codepage == ANSICP)
+    {
+      /* cfg_clipboard_type = CF_TEXT; */
+      return;
+    }
+  if (cfg_codepage == OEMCP)
+    {
+      cfg_clipboard_type = CF_OEMTEXT;
+      return;
+    }
 
-	/* Need to know final size after CR chars are inserted (the
-	   standard CF_TEXT clipboard format uses CRLF line endings,
-	   while Emacs uses just LF internally).  */
+  /* Else determine a suitable locale the hard way. */
+  EnumSystemLocales (enum_locale_callback, LCID_INSTALLED);
+}
 
-	truelen = nbytes + nlines;
+static BOOL WINAPI
+enum_locale_callback (/*const*/ char* loc_string)
+{
+  LCID lcid;
+  UINT codepage;
 
-	if ((htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, truelen)) == NULL)
-	  goto error;
+  lcid = strtoul (loc_string, NULL, 16);
 
-	if ((dst = (unsigned char *) GlobalLock (htext)) == NULL)
-	  goto error;
+  /* Is the wanted codepage the "ANSI" codepage for this locale? */
+  codepage = cp_from_locale (lcid, CF_TEXT);
+  if (codepage == cfg_codepage)
+    {
+      cfg_lcid = lcid;
+      cfg_clipboard_type = CF_TEXT;
+      return FALSE; /* Stop enumeration */
+    }
+  
+  /* Is the wanted codepage the OEM codepage for this locale? */
+  codepage = cp_from_locale (lcid, CF_OEMTEXT);
+  if (codepage == cfg_codepage)
+    {
+      cfg_lcid = lcid;
+      cfg_clipboard_type = CF_OEMTEXT;
+      return FALSE; /* Stop enumeration */
+    }
 
-	/* convert to CRLF line endings expected by clipboard */
-	while (1)
-	  {
-	    unsigned char *next;
-	    /* copy next line or remaining bytes including '\0' */
-	    next = _memccpy (dst, src, '\n', nbytes);
-	    if (next)
-	      {
-		/* copied one line ending with '\n' */
-		int copied = next - dst;
-		nbytes -= copied;
-		src += copied;
-		/* insert '\r' before '\n' */
-		next[-1] = '\r';
-		next[0] = '\n';
-		dst = next + 1;
-	      }
-	    else
-	      /* copied remaining partial line -> now finished */
-	      break;
-	  }
+  return TRUE; /* Continue enumeration */
+}
 
-	GlobalUnlock (htext);
+static UINT
+cp_from_locale (LCID lcid, UINT format)
+{
+  char buffer[20] = "";
+  UINT variant, cp;
 
-	Vlast_coding_system_used = Qraw_text;
-      }
-    else
-      {
-	/* We must encode contents of OBJ to the selection coding
-           system. */
-	int bufsize;
-	struct coding_system coding;
-	HANDLE htext2;
+  variant =
+    format == CF_TEXT ? LOCALE_IDEFAULTANSICODEPAGE : LOCALE_IDEFAULTCODEPAGE;
 
-	if (NILP (Vnext_selection_coding_system))
-	  Vnext_selection_coding_system = Vselection_coding_system;
-	setup_coding_system
-	  (Fcheck_coding_system (Vnext_selection_coding_system), &coding);
-	if (SYMBOLP (coding.pre_write_conversion)
-	    && !NILP (Ffboundp (coding.pre_write_conversion)))
-	  {
-	    string = run_pre_post_conversion_on_str (string, &coding, 1);
-	    src = SDATA (string);
-	    /* Include the terminating NULL character in the source of
-	       conversion.  */
-	    nbytes = SBYTES (string) + 1;
-	  }
-	coding.src_multibyte = 1;
-	coding.dst_multibyte = 0;
-	/* Need to set COMPOSITION_DISABLED, otherwise Emacs crashes in
-	   encode_coding_iso2022 trying to dereference a null pointer.  */
-	coding.composing = COMPOSITION_DISABLED;
-	if (coding.type == coding_type_iso2022)
-	  coding.flags |= CODING_FLAG_ISO_SAFE;
-	Vnext_selection_coding_system = Qnil;
-	coding.mode |= CODING_MODE_LAST_BLOCK;
-	bufsize = encoding_buffer_size (&coding, nbytes);
-	if ((htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, bufsize)) == NULL)
-	  goto error;
-	if ((dst = (unsigned char *) GlobalLock (htext)) == NULL)
-	  goto error;
-	encode_coding (&coding, src, dst, nbytes, bufsize);
-	Vlast_coding_system_used = coding.symbol;
+  GetLocaleInfo (lcid, variant, buffer, sizeof (buffer));
+  cp = strtoul (buffer, NULL, 10);
 
-	/* If clipboard sequence numbers are not supported, keep a copy for
-	   later comparison.  */
-	if (!clipboard_sequence_fn)
-	  {
-	    /* Stash away the data we are about to put into the
-	       clipboard, so we could later check inside
-	       Fw32_get_clipboard_data whether the clipboard still
-	       holds our data.  */
-	    if (clipboard_storage_size < coding.produced)
-	      {
-		clipboard_storage_size = coding.produced + 100;
-		last_clipboard_text = (char *) xrealloc (last_clipboard_text,
-							 clipboard_storage_size);
-	      }
-	    if (last_clipboard_text)
-	      memcpy (last_clipboard_text, dst, coding.produced);
-	  }
+  if (cp == CP_ACP)
+    return ANSICP;
+  else if (cp == CP_OEMCP)
+    return OEMCP;
+  else
+    return cp;
+}
 
-	GlobalUnlock (htext);
+static Lisp_Object
+coding_from_cp (UINT codepage)
+{
+  char buffer[30];
+  sprintf (buffer, "cp%d-dos", (int) codepage);
+  return intern (buffer);
+  /* We don't need to check that this coding system exists right here,
+     because that is done when the coding system is actually
+     instantiated, i.e. it is passed through Fcheck_coding_system()
+     there.  */
+}
 
-	/* Shrink data block to actual size.  */
-	htext2 = GlobalReAlloc (htext, coding.produced,
-                                GMEM_MOVEABLE | GMEM_DDESHARE);
-	if (htext2 != NULL) htext = htext2;
-      }
-  }
 
-  if (!OpenClipboard ((!NILP (frame) && FRAME_W32_P (XFRAME (frame))) ? FRAME_W32_WINDOW (XFRAME (frame)) : NULL))
+DEFUN ("w32-set-clipboard-data", Fw32_set_clipboard_data,
+       Sw32_set_clipboard_data, 1, 2, 0,
+       doc: /* This sets the clipboard data to the given text.  */)
+    (string, ignored)
+    Lisp_Object string, ignored;
+{
+  BOOL ok = TRUE;
+  HGLOBAL hlocale = NULL;
+  int nbytes;
+  unsigned char *src;
+  unsigned char *dst;
+  unsigned char *end;
+
+  (void) ignored;
+
+  CHECK_STRING (string);
+
+  setup_config ();
+
+  current_text = string;
+  current_coding_system = cfg_coding_system;
+  current_clipboard_type = cfg_clipboard_type;
+  current_lcid = cfg_lcid;
+  current_num_nls = 0;
+  current_requires_encoding = 0;
+  
+  BLOCK_INPUT;
+
+  /* Check for non-ASCII characters.  While we are at it, count the
+     number of LFs, so we know how many CRs we will have to add later
+     (just in the case where we can use our internal ASCII rendering,
+     see code and note in convert_to_handle_as_ascii() above).  */
+  nbytes = SBYTES (string);
+  src = SDATA (string);
+
+  for (dst = src, end = src+nbytes; dst < end; dst++)
+    {
+      if (*dst == '\n')
+	current_num_nls++;
+      else if (*dst >= 0x80 || *dst == 0)
+	{
+	  current_requires_encoding = 1;
+	  break;
+	}
+    }
+
+  if (!current_requires_encoding)
+    {
+      /* If all we have is ASCII we don't need to pretend we offer
+	 anything fancy. */
+      current_coding_system = Qraw_text;
+      current_clipboard_type = CF_TEXT;
+    }
+  else
+    {
+      /* If we have something non-ASCII we may want to set a locale.
+	 We always do that directly (non-delayed), as it's just a
+	 small bit and it simplifies the rest of the code. */
+      if (cfg_lcid != LOCALE_NEUTRAL && cfg_lcid != DEFAULT_LCID)
+	{
+	  hlocale = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE,
+				 sizeof (cfg_lcid));
+	  if (hlocale != NULL)
+	    {
+	      LCID * lcid_ptr;
+	      if ((lcid_ptr = (LCID *) GlobalLock (hlocale)) != NULL)
+		{
+		  *lcid_ptr = cfg_lcid;
+		  GlobalUnlock (hlocale);
+		}
+	    }
+	}
+    }
+
+  if (!OpenClipboard (clipboard_owner))
     goto error;
 
-  ok = EmptyClipboard () && SetClipboardData (CF_TEXT, htext);
+  ++modifying_clipboard;
+  ok = EmptyClipboard ();
+  --modifying_clipboard;
+
+  if (ok)
+    {
+      if (clipboard_owner == NULL)
+	{
+	  /* If for some reason we don't have a clipboard_owner, we
+	     just set the text format as chosen by the configuration
+	     and than forget about the whole thing.  */
+	  render (make_number (current_clipboard_type));
+	  current_text = Qnil;
+	  current_coding_system = Qnil;
+	}
+      else
+	{
+	  /* Advertise all supported formats so that whatever the
+	     requester chooses, only one encoding step needs to be
+	     made.  This is intentionally different from what we do in
+	     the WM_RENDERALLFORMATS handler.  */
+	  SetClipboardData (CF_UNICODETEXT, NULL);
+	  SetClipboardData (CF_TEXT, NULL);
+	  SetClipboardData (CF_OEMTEXT, NULL);
+	}
+
+      if (hlocale != NULL)
+	{
+	  ok = SetClipboardData (CF_LOCALE, hlocale) != NULL;
+	  if (!ok) EmptyClipboard ();
+	}
+    }
 
   CloseClipboard ();
 
-  /* Common sense says to read the sequence number inside the
-     OpenClipboard/ CloseClipboard block to avoid race conditions
-     where another app puts something on the clipboard straight after
-     us. But experience suggests that the sequence number from the
-     SetClipboardData is not allocated until we close the clipboard!
-     Since clipboard operations are normally user-driven, the race
-     condition is probably not going to really happen.  */
-  if (clipboard_sequence_fn)
-    last_clipboard_sequence_number = clipboard_sequence_fn ();
+  /* We haven't really "used" this coding system yet, and it's even
+     unclear if we ever will.  But this is a way to tell the upper
+     level what we *would* use under ideal circumstances.
+
+     We don't signal the actually used coding-system later when we
+     finally render, because that can happen at any time and we don't
+     want to disturb the "foreground" action. */
+  Vlast_coding_system_used = current_coding_system;
+
+  Vnext_selection_coding_system = Qnil;
 
   if (ok) goto done;
 
  error:
 
   ok = FALSE;
-  if (htext) GlobalFree (htext);
-  if (last_clipboard_text)
-    *last_clipboard_text = '\0';
+  if (hlocale) GlobalFree (hlocale);
 
-  last_clipboard_sequence_number = 0;
+  current_text = Qnil;
+  current_coding_system = Qnil;
 
  done:
   UNBLOCK_INPUT;
@@ -292,24 +700,50 @@ DEFUN ("w32-set-clipboard-data", Fw32_se
   return (ok ? string : Qnil);
 }
 
+
 DEFUN ("w32-get-clipboard-data", Fw32_get_clipboard_data,
        Sw32_get_clipboard_data, 0, 1, 0,
        doc: /* This gets the clipboard data in text format.  */)
-     (frame)
-     Lisp_Object frame;
+     (ignored)
+     Lisp_Object ignored;
 {
-  HANDLE htext;
+  HGLOBAL htext;
   Lisp_Object ret = Qnil;
+  UINT actual_clipboard_type;
+  int use_configured_coding_system = 1;
+
+  (void) ignored;
 
-  if (!NILP (frame))
-    CHECK_LIVE_FRAME (frame);
+  /* Don't pass our own text from the clipboard (which might be
+     troublesome if the killed text includes null characters).  */
+  if (!NILP (current_text))
+    return ret;
+
+  setup_config ();
+  actual_clipboard_type = cfg_clipboard_type;
 
   BLOCK_INPUT;
 
-  if (!OpenClipboard ((!NILP (frame) && FRAME_W32_P (XFRAME (frame))) ? FRAME_W32_WINDOW (XFRAME (frame)) : NULL))
+  if (!OpenClipboard (clipboard_owner))
     goto done;
 
-  if ((htext = GetClipboardData (CF_TEXT)) == NULL)
+  if ((htext = GetClipboardData (actual_clipboard_type)) == NULL)
+    {
+      /* If we want CF_UNICODETEXT but can't get it, the current
+	 coding system is useless.  OTOH we can still try and decode
+	 CF_TEXT based on the locale that the system gives us and that
+	 we get down below.  */
+      if (actual_clipboard_type == CF_UNICODETEXT)
+	{
+	  htext = GetClipboardData (CF_TEXT);
+	  if (htext != NULL)
+	    {
+	      actual_clipboard_type = CF_TEXT;
+	      use_configured_coding_system = 0;
+	    }
+	}
+    }
+  if (htext == NULL)
     goto closeclip;
 
   {
@@ -322,53 +756,107 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
     if ((src = (unsigned char *) GlobalLock (htext)) == NULL)
       goto closeclip;
 
-    nbytes = strlen (src);
-
-    /* If the text in clipboard is identical to what we put there
-       last time w32_set_clipboard_data was called, pretend there's no
-       data in the clipboard.  This is so we don't pass our own text
-       from the clipboard (which might be troublesome if the killed
-       text includes null characters).  */
-    if ((clipboard_sequence_fn
-	 && clipboard_sequence_fn () == last_clipboard_sequence_number)
-	|| (last_clipboard_text
-	    && clipboard_storage_size >= nbytes
-	    && memcmp(last_clipboard_text, src, nbytes) == 0))
-      goto closeclip;
+    /* If the clipboard data contains any non-ascii code, we need to
+       decode it with a coding system.
+       FIXME: Repeat the code for the Unicode case. */
+    if (actual_clipboard_type == CF_UNICODETEXT)
+      {
+	nbytes = lstrlenW ((WCHAR *)src) * 2;
+	require_decoding = 1;
+      }
+    else
+      {
+	int i;
 
-    {
-      /* If the clipboard data contains any non-ascii code, we
-	 need to decode it.  */
-      int i;
+	nbytes = strlen (src);
 
-      for (i = 0; i < nbytes; i++)
-	{
-	  if (src[i] >= 0x80)
-	    {
-	      require_decoding = 1;
-	      break;
-	    }
-	}
-    }
+	for (i = 0; i < nbytes; i++)
+	  {
+	    if (src[i] >= 0x80)
+	      {
+		require_decoding = 1;
+		break;
+	      }
+	  }
+      }
 
     if (require_decoding)
       {
 	int bufsize;
 	unsigned char *buf;
 	struct coding_system coding;
+	Lisp_Object coding_system = Qnil;
+	
+	/* `next-selection-coding-system' should override everything,
+	   even when the locale passed by the system disagrees.  The
+	   only exception is when `next-selection-coding-system'
+	   requested CF_UNICODETEXT and we couldn't get that. */
+	if (use_configured_coding_system
+	    && !NILP (Vnext_selection_coding_system))
+	    coding_system = Vnext_selection_coding_system;
+
+	/* If we have CF_TEXT or CF_OEMTEXT, we want to check out
+	   CF_LOCALE, too. */
+	else if (actual_clipboard_type != CF_UNICODETEXT)
+	  {
+	    HGLOBAL hlocale;
+	    LCID lcid = DEFAULT_LCID;
+	    UINT cp;
+
+	    /* Windows is documented to always generate CF_LOCALE info
+	       automatically, so the locale handle should be always
+	       present, but this is not always true on 9x ;-(.  */
+	    hlocale = GetClipboardData (CF_LOCALE);
+	    if (hlocale != NULL)
+	      {
+		const LCID * lcid_ptr;
+		lcid_ptr = (const LCID *) GlobalLock (hlocale);
+		if (lcid_ptr != NULL)
+		  {
+		    lcid = *lcid_ptr;
+		    GlobalUnlock (hlocale);
+		  }
+
+		/* 9x has garbage as the sort order (to be exact there
+		   is another instance of the language id in the upper
+		   word).  We don't care about sort order anyway, so
+		   we can just filter out the unneeded mis-information
+		   to avoid irritations. */
+		lcid = MAKELCID (LANGIDFROMLCID (lcid), SORT_DEFAULT);
+	      }
 
-	if (NILP (Vnext_selection_coding_system))
-	  Vnext_selection_coding_system = Vselection_coding_system;
-	setup_coding_system
-	  (Fcheck_coding_system (Vnext_selection_coding_system), &coding);
+	    /* If we are using fallback from CF_UNICODETEXT, we can't
+	       use the configured coding system.  Also we don't want
+	       to use it, if the system has supplied us with a locale
+	       and it is not just the system default. */
+	    if (!use_configured_coding_system || lcid != DEFAULT_LCID)
+	      {
+		cp = cp_from_locale (lcid, actual_clipboard_type);
+		/* If it's just our current standard setting anyway,
+		   use the coding system that the user has selected.
+		   Otherwise create a new spec to match the locale
+		   that was specified by the other side or the
+		   system.  */
+		if (!use_configured_coding_system || cp != cfg_codepage)
+		  coding_system = coding_from_cp (cp);
+	      }
+	  }
+
+	if (NILP (coding_system))
+	  coding_system = Vselection_coding_system;
+	Vnext_selection_coding_system = Qnil;
+
+	setup_coding_system (Fcheck_coding_system (coding_system), &coding);
 	coding.src_multibyte = 0;
 	coding.dst_multibyte = 1;
-	Vnext_selection_coding_system = Qnil;
 	coding.mode |= CODING_MODE_LAST_BLOCK;
 	/* We explicitely disable composition handling because
 	   selection data should not contain any composition
 	   sequence.  */
 	coding.composing = COMPOSITION_DISABLED;
+	/* Force DOS line-ends. */
+	coding.eol_type = CODING_EOL_CRLF;
+
 	bufsize = decoding_buffer_size (&coding, nbytes);
 	buf = (unsigned char *) xmalloc (bufsize);
 	decode_coding (&coding, src, buf, nbytes, bufsize);
@@ -382,10 +870,10 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
       }
     else
       {
-	/* Need to know final size after CR chars are removed because we
-	   can't change the string size manually, and doing an extra
-	   copy is silly.  Note that we only remove CR when it appears
-	   as part of CRLF.  */
+	/* Need to know final size after CR chars are removed because
+	   we can't change the string size manually, and doing an
+	   extra copy is silly.  We only remove CR when it appears as
+	   part of CRLF.  */
 
 	truelen = nbytes;
 	dst = src;
@@ -462,9 +950,14 @@ and t is the same as `SECONDARY'.  */)
 
       if (OpenClipboard (NULL))
 	{
-	  int format = 0;
-	  while (format = EnumClipboardFormats (format))
-	    if (format == CF_TEXT)
+	  UINT format = 0;
+	  setup_config ();
+	  while ((format = EnumClipboardFormats (format)))
+	    /* Check CF_TEXT in addition to cfg_clipboard_type,
+	       because we can fall back on that if CF_UNICODETEXT is
+	       not available.  Actually a check for CF_TEXT only
+	       should be enough.  */
+	    if (format == cfg_clipboard_type || format == CF_TEXT)
 	      {
 		val = Qt;
 		break;
@@ -476,24 +969,25 @@ and t is the same as `SECONDARY'.  */)
   return Qnil;
 }
 
+/* One-time init.  Called in the un-dumped Emacs, but not in the
+   dumped version. */
+
 void
 syms_of_w32select ()
 {
-#if 0
-  defsubr (&Sw32_open_clipboard);
-  defsubr (&Sw32_empty_clipboard);
-  defsubr (&Sw32_close_clipboard);
-#endif
   defsubr (&Sw32_set_clipboard_data);
   defsubr (&Sw32_get_clipboard_data);
   defsubr (&Sx_selection_exists_p);
 
   DEFVAR_LISP ("selection-coding-system", &Vselection_coding_system,
 	       doc: /* Coding system for communicating with other programs.
-When sending or receiving text via cut_buffer, selection, and clipboard,
-the text is encoded or decoded by this coding system.
-The default value is `iso-latin-1-dos'.  */);
-  Vselection_coding_system = intern ("iso-latin-1-dos");
+When sending or receiving text via cut_buffer, selection, and
+clipboard, the text is encoded or decoded by this coding system.
+The default value is the current system default encoding on 9x/Me and
+`utf-16le-dos' (Unicode) on NT/W2K/XP. */);  
+  /* The actual value is set dynamically in the dumped Emacs, see
+     below. */
+  Vselection_coding_system = Qnil;
 
   DEFVAR_LISP ("next-selection-coding-system", &Vnext_selection_coding_system,
 	       doc: /* Coding system for the next communication with other programs.
@@ -504,6 +998,41 @@ set to nil.  */);
   Vnext_selection_coding_system = Qnil;
 
   QCLIPBOARD = intern ("CLIPBOARD");	staticpro (&QCLIPBOARD);
+
+  cfg_coding_system = Qnil;     staticpro (&cfg_coding_system);
+  current_text = Qnil;		staticpro (&current_text);
+  current_coding_system = Qnil; staticpro (&current_coding_system);
+
+  QUNICODE = intern ("utf-16le-dos"); staticpro (&QUNICODE);
+  QANSICP = Qnil; staticpro (&QANSICP);
+  QOEMCP = Qnil;  staticpro (&QOEMCP);
+}
+
+/* One-time init.  Called in the dumped Emacs, but not in the
+   un-dumped version. */
+
+void
+globals_of_w32select ()
+{
+  DEFAULT_LCID = GetUserDefaultLCID ();
+  /* Drop the sort order from the LCID, so we can compare this with
+     CF_LOCALE objects that have the same fix on 9x.  */
+  DEFAULT_LCID = MAKELCID (LANGIDFROMLCID (DEFAULT_LCID), SORT_DEFAULT);
+
+  ANSICP = GetACP ();
+  OEMCP = GetOEMCP ();
+
+  QANSICP = coding_from_cp (ANSICP);
+  QOEMCP = coding_from_cp (OEMCP);
+
+  if (os_subtype == OS_NT)
+    Vselection_coding_system = QUNICODE;
+  else if (inhibit_window_system)
+    Vselection_coding_system = QOEMCP;
+  else
+    Vselection_coding_system = QANSICP;
+
+  clipboard_owner = create_owner ();
 }
 
 /* arch-tag: c96e9724-5eb1-4dad-be07-289f092fd2af

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Unicode support for the MS Windows clipboard

[Jason Rumney]

[-- Attachment #1.1: Type: text/plain, Size: 578 bytes --]

Benjamin Riefenstahl wrote:

>Hi Jason,
>
>Jason Rumney <jasonr@gnu.org> writes:
>  
>
>>Make sure that rendering CF_TEXT does not clobber the CF_UNICODETEXT
>>that you just put there on the NT based versions of Windows that do
>>automatic conversions. It might be better to reverse the order.
>>    
>>
>
>It's not a problem on W2K here and thinking about it, it would be a
>bug in the OS if it was, automatic conversion is supposed to be a
>fallback solution.
>

I thought that would be the case, but wasn't sure so I just wanted to 
make sure you'd considered it.
Good work.

[-- Attachment #1.2: Type: text/html, Size: 1082 bytes --]

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

Hi Jason,


> Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:
>> The uncertainty about the right Unicode coding-system to use seems
>> to me to be a general problem with the naming of the Unicode
>> coding-systems, not with w32select.c.

Jason Rumney <jasonr@gnu.org> writes:
> My point was that users shouldn't have to know the differences
> between little-endian and big-endian and with/without signatures
> just to use the Windows clipboard.

IMO users shouldn't have to know anything about coding systems in
general (and the fact is probably that they usually don't).  But
shielding them from that should be the job of some code level further
up the food chain.

> No matter what we call those coding systems, users will still need
> intimate knowledge of Unicode to know what they mean.

a) I suggested adding a system specific coding-system alias "unicode"
in the text that you snipped.  ;-)

b) It seems cleaner to me to make a pretty customize interface that
just selects between clipboard modes "Unicode" and "8-bit" and does
the right thing with that user input.  One could also add symbols like
this as special cases to set-selection-coding-system.


benny

Re: Unicode support for the MS Windows clipboard

[Tak Ota]

This patch introduces the following linking problem.

        link -out:obj/i386/temacs.bin -nologo -release -incremental:no -version:3.10 -swaprun:cd -swaprun:net setargv.obj -debug:full -debugtype:both  -stack:0x00800000 -heap:0x00100000 -base:0x01000000 -debug:full -debugtype:both -pdb:obj/i386\temacs.pdb -machine:i386 -subsystem:console -entry:_start -map:obj/i386\temacs.map -profile obj/i386/firstfile.obj obj/i386/emacs.res obj/i386/temacs0.lib  obj/i386/temacs1.lib        obj/i386/temacw32.lib   obj/i386/lastfile.lib   winmm.lib     advapi32.lib      gdi32.lib       comdlg32.lib    user32.lib      mpr.lib             shell32.lib     winspool.lib    libc.lib
libc.lib(heapinit.obj) : error LNK2005: __heap_init already defined in temacs1.lib(w32heap.obj)
libc.lib(heapinit.obj) : error LNK2005: __heap_term already defined in temacs1.lib(w32heap.obj)
obj/i386/temacs.bin : fatal error LNK1169: one or more multiply defined symbols found

-Tak

Tue, 27 Jul 2004 14:24:14 +0200: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> wrote:

> Hi Jason,
> 
> Jason Rumney <jasonr@gnu.org> writes:
> > Make sure that rendering CF_TEXT does not clobber the CF_UNICODETEXT
> > that you just put there on the NT based versions of Windows that do
> > automatic conversions. It might be better to reverse the order.
> 
> It's not a problem on W2K here and thinking about it, it would be a
> bug in the OS if it was, automatic conversion is supposed to be a
> fallback solution.  But better be safe than sorry.  See attached
> revised patch.
> 
> benny
> 
> 

Re: Unicode support for the MS Windows clipboard

[Eli Zaretskii]

> From: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de>
> Date: Wed, 28 Jul 2004 00:43:12 +0200
> 
> a) I suggested adding a system specific coding-system alias "unicode"
> in the text that you snipped.  ;-)
> 
> b) It seems cleaner to me to make a pretty customize interface that
> just selects between clipboard modes "Unicode" and "8-bit" and does
> the right thing with that user input.  One could also add symbols like
> this as special cases to set-selection-coding-system.

Right.  The Unicode mode of the Windows clipboard is not really a
coding-system, as it cannot be used in any other context.  Therefore,
I think we should have a w32-clipboard-encoding-mode or some such, not
a coding-system.

Re: Unicode support for the MS Windows clipboard

[Jason Rumney]

"Eli Zaretskii" <eliz@gnu.org> writes:

> Right.  The Unicode mode of the Windows clipboard is not really a
> coding-system, as it cannot be used in any other context.  Therefore,
> I think we should have a w32-clipboard-encoding-mode or some such, not
> a coding-system.

I don't see any point in introducing a new variable that will become
obsolete in the next version of Emacs. With Unicode as the internal
encoding of Emacs, it will always be better to use unicode-mode, so
there is no point in confusing users with the option. Even now, I
cannot think of any coding-systems that are supported by Windows that
we do not have Unicode support for. Since we originally discussed
Unicode clipboard support, the CJK unicode tables have been made to
autoload on demand, so the last remaining hurdle for making Unicode
the default on Windows is gone.

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

Hi Tak,


Tak Ota <Takaaki.Ota@am.sony.com> writes:
> This patch introduces the following linking problem.

Thanks for trying it out ;-)

> libc.lib(heapinit.obj) : error LNK2005: __heap_init already defined in
> temacs1.lib(w32heap.obj)

The patch shouldn't have touched those two.  The only connection is
that w32select.c uses "os_subtype", which I is defined in w32heap.c.
For that w32select.c includes "w32heap.h", that is all the conneciton
there is.


benny

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

Hi Eli,


"Eli Zaretskii" <eliz@gnu.org> writes:
> Right.  The Unicode mode of the Windows clipboard is not really a
> coding-system, as it cannot be used in any other context.

I don't get you?  utf-16le is a regular coding system that certainly
can be used in any other context.  It is not special at all, quite the
contrary, it is more or less basic.

> Therefore, I think we should have a w32-clipboard-encoding-mode or
> some such, not a coding-system.

We could decide not to use selection-coding-system at all.  In that
case we probably don't need any variable at all, just trusting the
system to give us the right information.

It was my understanding though, that some localized versions of
Windows may need this kind of configuration.  I know for sure that
there used to be wide-spread versions of Windows in Russia, which were
modified and in which encoding didn't work regularly.  This was ten
years ago, so I don't know what the state of things is today.


benny

Re: Unicode support for the MS Windows clipboard

[Jason Rumney]

[-- Attachment #1.1: Type: text/plain, Size: 927 bytes --]

Benjamin Riefenstahl wrote:

>Hi Tak,
>
>
>Tak Ota <Takaaki.Ota@am.sony.com> writes:
>  
>
>>This patch introduces the following linking problem.
>>    
>>
>
>Thanks for trying it out ;-)
>
>  
>
>>libc.lib(heapinit.obj) : error LNK2005: __heap_init already defined in
>>temacs1.lib(w32heap.obj)
>>    
>>
>
>The patch shouldn't have touched those two.  The only connection is
>that w32select.c uses "os_subtype", which I is defined in w32heap.c.
>For that w32select.c includes "w32heap.h", that is all the conneciton
>there is.
>  
>
If this is the source of the problem, then moving os_subtype to w32.h 
might fix it. But first, we must understand why this problem has 
occurred. Is the new implementation of w32select.c using a C library 
function that internally mallocs memory using the C library malloc 
instead of Emacs' one? Or is it the calls to GlobalAlloc that is causing 
this (in which case there is no conflict).

[-- Attachment #1.2: Type: text/html, Size: 1513 bytes --]

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

Hi Jason,


>>Tak Ota <Takaaki.Ota@am.sony.com> writes:
>>>libc.lib(heapinit.obj) : error LNK2005: __heap_init already defined in
>>>temacs1.lib(w32heap.obj)

Jason Rumney <jasonr@gnu.org> writes:
> Is the new implementation of w32select.c using a C library function
> that internally mallocs memory using the C library malloc instead of
> Emacs' one?

Ah, now I understand.  The code is using atexit(), os_subtype is a red
herring.

We are talking about MSC and static linking and the problem with
atexit() mentioned in w32.c.  I commented on that in the code.  The
problem can't occur in a dynamic build, which is all that Mingw (my
environment) supports.

So how do I simulate atexit()?  The method using signals as used in
w32.c doesn't seem to work with Mingw, I tried that.


benny

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:
> We could decide not to use selection-coding-system at all.  In that
> case we probably don't need any variable at all, just trusting the
> system to give us the right information.

On second thought, that would also loose the ability to specify a
non-default encoding for 8-bit text.  Although I doubt that many
receiving applications use that correctly :-(.  And those that could
use it will probably all prefer a Unicode version when we offer that
in addition.


benny

Re: Unicode support for the MS Windows clipboard

[Jason Rumney]

[-- Attachment #1.1: Type: text/plain, Size: 1620 bytes --]

Benjamin Riefenstahl wrote:

>Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:
>  
>
>>We could decide not to use selection-coding-system at all.  In that
>>case we probably don't need any variable at all, just trusting the
>>system to give us the right information.
>>    
>>
>
>On second thought, that would also loose the ability to specify a
>non-default encoding for 8-bit text.  Although I doubt that many
>receiving applications use that correctly :-(.  And those that could
>use it will probably all prefer a Unicode version when we offer that
>in addition.
>  
>
Here are my suggestions, which should make things work the same way as 
all other Windows apps by default, and any changes to 
selection-coding-system would be for specialised uses (ie, for use with 
other software that does something unusual) so it would not be 
unreasonable to expect users to either know what they are doing or have 
a precise recipe to follow.


Default selection-coding-system to utf-16-le (or utf-16le, whichever is 
appropriate).

When selection-coding-system is utf-16-le, use CF_UNICODETEXT, and put a 
copy on CF_TEXT in the system's default encoding for Windows 9x apps 
that do not understand Unicode.

If the user changes selection-coding-system, use only CF_TEXT or 
CF_OEMTEXT, depending on the coding-system specified (codepage < 900 
should be a good enough test I think, but double check there are not 
exceptions).  When putting data on the clipboard as CF_TEXT, we should 
try to set an appropriate CF_LOCALE if we can determine it, so that 
automatic conversion to CF_UNICODETEXT works correctly.


[-- Attachment #1.2: Type: text/html, Size: 2163 bytes --]

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

Hi Jason,


Jason Rumney <jasonr@gnu.org> writes:
> Here are my suggestions,

?? 

> Default selection-coding-system to utf-16-le (or utf-16le, whichever
> is appropriate).

Already done on NT.  So you just want to do that on 9x, too?

> When selection-coding-system is utf-16-le, use CF_UNICODETEXT, and
> put a copy on CF_TEXT in the system's default encoding for Windows
> 9x apps that do not understand Unicode.

Already done (delayed rendering makes that work almost automatically).

> If the user changes selection-coding-system, use only CF_TEXT or
> CF_OEMTEXT,

Already done. 

> When putting data on the clipboard as CF_TEXT, we should try to set
> an appropriate CF_LOCALE if we can determine it, so that automatic
> conversion to CF_UNICODETEXT works correctly.

Already done.


I must be missing something.   What was the point here?


benny

Re: Unicode support for the MS Windows clipboard

[Tak Ota]

Wed, 28 Jul 2004 12:35:15 +0100: Jason Rumney <jasonr@gnu.org> wrote:

> > > libc.lib(heapinit.obj) : error LNK2005: __heap_init already defined in
> > > temacs1.lib(w32heap.obj)
> > 
> > 
> > The patch shouldn't have touched those two.  The only connection is
> > that w32select.c uses "os_subtype", which I is defined in w32heap.c.
> > For that w32select.c includes "w32heap.h", that is all the conneciton
> > there is.
>   
> 
> If this is the source of the problem, then moving os_subtype to w32.h
> might fix it. But first, we must understand why this problem has
> occurred. Is the new implementation of w32select.c using a C library
> function that internally mallocs memory using the C library malloc
> instead of Emacs' one? Or is it the calls to GlobalAlloc that is causing
> this (in which case there is no conflict).

I already experimented not to including w32headp.c but locally define
enum {OS_OSWIN95 = 1, OS_NT }; and extern int os_subtype; and found
the compilation passed but the same link error occurred.

Normally (without Benjamin's patch) _heap_init and _heap_term are
provided by w32heap.c but the patch somehow pulls in these function
from the MS library I suppose.  I suspect Jason's theory of hidden
malloc maybe the case.

I'll continue to investigate.

-Tak

Re: Unicode support for the MS Windows clipboard

[Tak Ota]

Benny and Jason,

Please disregard my previous message.  I replied before reading this
one.  So now we know the problem.  I'll leave the solution to you w32
experts.  Thanks.

-Tak

Wed, 28 Jul 2004 14:08:07 +0200: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> wrote:

> Hi Jason,
> 
> 
> >>Tak Ota <Takaaki.Ota@am.sony.com> writes:
> >>>libc.lib(heapinit.obj) : error LNK2005: __heap_init already defined in
> >>>temacs1.lib(w32heap.obj)
> 
> Jason Rumney <jasonr@gnu.org> writes:
> > Is the new implementation of w32select.c using a C library function
> > that internally mallocs memory using the C library malloc instead of
> > Emacs' one?
> 
> Ah, now I understand.  The code is using atexit(), os_subtype is a red
> herring.
> 
> We are talking about MSC and static linking and the problem with
> atexit() mentioned in w32.c.  I commented on that in the code.  The
> problem can't occur in a dynamic build, which is all that Mingw (my
> environment) supports.
> 
> So how do I simulate atexit()?  The method using signals as used in
> w32.c doesn't seem to work with Mingw, I tried that.
> 
> 
> benny

Re: Unicode support for the MS Windows clipboard

[Tak Ota]

Wed, 28 Jul 2004 14:08:07 +0200: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> wrote:

> Hi Jason,
> 
> 
> >>Tak Ota <Takaaki.Ota@am.sony.com> writes:
> >>>libc.lib(heapinit.obj) : error LNK2005: __heap_init already defined in
> >>>temacs1.lib(w32heap.obj)
> 
> Jason Rumney <jasonr@gnu.org> writes:
> > Is the new implementation of w32select.c using a C library function
> > that internally mallocs memory using the C library malloc instead of
> > Emacs' one?
> 
> Ah, now I understand.  The code is using atexit(), os_subtype is a red
> herring.
> 
> We are talking about MSC and static linking and the problem with
> atexit() mentioned in w32.c.  I commented on that in the code.  The
> problem can't occur in a dynamic build, which is all that Mingw (my
> environment) supports.
> 
> So how do I simulate atexit()?  The method using signals as used in
> w32.c doesn't seem to work with Mingw, I tried that.

Benny,

For you information, I confirmed term_ntproc() is actually called at
exit on MSVC build.

I think w32.c should provide emacs version of atexit() somehow.

-Tak

Re: Unicode support for the MS Windows clipboard

[Tak Ota]

Benny,

After removing atexit(), kludging atexit_callback() into term_ntproc()
and (set-clipboard-coding-system 'utf-16-le) first time ever I can
share Japanese text between emacs and other application programs
through the clipboard.  Thank you very much.  I look forward to the
final implementation coming soon.

-Tak

Tue, 27 Jul 2004 14:24:14 +0200: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> wrote:

> Hi Jason,
> 
> Jason Rumney <jasonr@gnu.org> writes:
> > Make sure that rendering CF_TEXT does not clobber the CF_UNICODETEXT
> > that you just put there on the NT based versions of Windows that do
> > automatic conversions. It might be better to reverse the order.
> 
> It's not a problem on W2K here and thinking about it, it would be a
> bug in the OS if it was, automatic conversion is supposed to be a
> fallback solution.  But better be safe than sorry.  See attached
> revised patch.
> 
> benny
> 
> 

Re: Unicode support for the MS Windows clipboard

[Eli Zaretskii]

> From: Jason Rumney <jasonr@gnu.org>
> Date: Wed, 28 Jul 2004 09:02:29 +0100
> 
> I don't see any point in introducing a new variable that will become
> obsolete in the next version of Emacs.

Sorry, I thought you actually supported such an option.  If you think
it's not needed, I won't argue.

Re: Unicode support for the MS Windows clipboard

[Tak Ota]

Benny,

After having applied your patch I noticed something strange.  There is
a loss of character when copying from w32 application to emacs.  The
very first character gets lost when going from w32 to emacs.  There is
no loss when copying from emacs and pasting in w32 application.  If I
cut/copy a single character in w32 application emacs doesn't even
notice the new clipboard data.  I am testing this on windoze XP pro.

-Tak

Wed, 28 Jul 2004 11:42:50 -0700 (PDT): Tak Ota <Takaaki.Ota@am.sony.com> wrote:

> Benny,
> 
> After removing atexit(), kludging atexit_callback() into term_ntproc()
> and (set-clipboard-coding-system 'utf-16-le) first time ever I can
> share Japanese text between emacs and other application programs
> through the clipboard.  Thank you very much.  I look forward to the
> final implementation coming soon.
> 
> -Tak
> 
> Tue, 27 Jul 2004 14:24:14 +0200: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> wrote:
> 
> > Hi Jason,
> > 
> > Jason Rumney <jasonr@gnu.org> writes:
> > > Make sure that rendering CF_TEXT does not clobber the CF_UNICODETEXT
> > > that you just put there on the NT based versions of Windows that do
> > > automatic conversions. It might be better to reverse the order.
> > 
> > It's not a problem on W2K here and thinking about it, it would be a
> > bug in the OS if it was, automatic conversion is supposed to be a
> > fallback solution.  But better be safe than sorry.  See attached
> > revised patch.
> > 
> > benny
> > 
> > 
> 
> 
> _______________________________________________
> Emacs-devel mailing list
> Emacs-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

Hi Tak,


Tak Ota <Takaaki.Ota@am.sony.com> writes:
>> After [...] (set-clipboard-coding-system 'utf-16-le)

> After having applied your patch I noticed something strange.  There
> is a loss of character when copying from w32 application to emacs.

That may be an effect of using 'utf-16-le.  Try 'utf-16le instead (no
dash before the "le").  Those two are slightly different, because
'utf-16-le adds and expects a byte-order-mark as first character.

> I am testing this on windoze XP pro.

Than 'utf-16le should be the default anyway.  You should be able to
just drop the call to set-clipboard-coding-system in your .emacs.


benny

Re: Unicode support for the MS Windows clipboard

[Tak Ota]

Benny,

You are right!  Leaving the clipboard-coding-system as the default
makes it work fine now.  Thank you very much.

-Tak

Thu, 29 Jul 2004 13:42:43 +0200: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> wrote:

> Hi Tak,
> 
> 
> Tak Ota <Takaaki.Ota@am.sony.com> writes:
> >> After [...] (set-clipboard-coding-system 'utf-16-le)
> 
> > After having applied your patch I noticed something strange.  There
> > is a loss of character when copying from w32 application to emacs.
> 
> That may be an effect of using 'utf-16-le.  Try 'utf-16le instead (no
> dash before the "le").  Those two are slightly different, because
> 'utf-16-le adds and expects a byte-order-mark as first character.
> 
> > I am testing this on windoze XP pro.
> 
> Than 'utf-16le should be the default anyway.  You should be able to
> just drop the call to set-clipboard-coding-system in your .emacs.
> 
> 
> benny

Re: Unicode support for the MS Windows clipboard

[Tak Ota]

Benny,

Are you going to complete this?  We know the use of atexit() is a
problem on MSVC build but otherwise it has been working fine for me.
In fact, this event triggered me to force myself to migrate to mingw
from MSVC.  I hope this unicode support gets checked in soon.

-Tak

Tue, 27 Jul 2004 14:24:14 +0200: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> wrote:

> Hi Jason,
> 
> Jason Rumney <jasonr@gnu.org> writes:
> > Make sure that rendering CF_TEXT does not clobber the CF_UNICODETEXT
> > that you just put there on the NT based versions of Windows that do
> > automatic conversions. It might be better to reverse the order.
> 
> It's not a problem on W2K here and thinking about it, it would be a
> bug in the OS if it was, automatic conversion is supposed to be a
> fallback solution.  But better be safe than sorry.  See attached
> revised patch.
> 
> benny
> 
> 

Re: Unicode support for the MS Windows clipboard

[Benjamin Riefenstahl]

Hi Tak,

Tak Ota <Takaaki.Ota@am.sony.com> writes:
> Are you going to complete this?  We know the use of atexit() is a
> problem on MSVC build but otherwise it has been working fine for me.
> In fact, this event triggered me to force myself to migrate to mingw
> from MSVC.  I hope this unicode support gets checked in soon.

I will fix the atexit() bit and see what I can do in terms of docs.
I'll probably send the result on Monday. 

benny

Re: Unicode support for the MS Windows clipboard [new patch]

[Benjamin Riefenstahl]

[-- Attachment #1: Type: text/plain, Size: 2275 bytes --]

Hi all,

Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:
> I will fix the atexit() bit and see what I can do in terms of docs.

See attached patch. 

I have kept the documentation rather high-level, because users
shouldn't need to do anything, the defaults are fine.  A more detailed
documentation would have to go into the ELisp manual IMO.

benny


2004-08-30  Benjamin Riefenstahl  <Benjamin.Riefenstahl@epost.de>

	* w32select.c: Summary: Thorough rework to implement Unicode
        clipboard operations and delayed rendering.

        Drop last_clipboard_text and related code, keep track of
        ownership via clipboard_owner instead.  Drop old #if0
        sections.

	(DEFAULT_LCID, ANSICP, OEMCP, QUNICODE, QANSICP, QOEMCP)
	(clipboard_owner, modifying_clipboard, cfg_coding_system)
	(cfg_codepage, cfg_lcid, cfg_clipboard_type, current_text)
	(current_coding_system, current_requires_encoding)
	(current_num_nls, current_clipboard_type, current_lcid): New
	static variables.

	(convert_to_handle_as_ascii, convert_to_handle_as_coded)
        (render, render_all, run_protected, lisp_error_handler)
        (owner_callback, create_owner, setup_config)
        (enum_locale_callback, cp_from_locale, coding_from_cp): New
        local functions.

        (term_w32select, globals_of_w32select): New global functions.

	(Fw32_set_clipboard_data): Ignore parameter FRAME, use
	clipboard_owner instead.  Use delayed rendering and provide
	all text formats.  Provide CF_LOCALE if necessary.

	(Fw32_get_clipboard_data): Handle CF_UNICODETEXT and
	CF_LOCALE.  Fall back to CF_TEXT, if CF_UNICODETEXT is not
	available.  Force DOS line-ends for decoding.

	(Fx_selection_exists_p): Handle CF_UNICODETEXT.

	(syms_of_w32select): Init and register new variables.

        * w32.h: Add prototypes for globals_of_w32select and
        term_w32select.  Make the neighboring K&R declarations into
        prototypes, too.

        * emacs.c: Include w32.h to get function prototypes.
        (main): Call globals_of_w32select.

        * w32.c (term_ntproc): Call term_w32select. 

        * mule-cmds.el (set-locale-environment): Remove call to
        set-selection-coding-system on Windows.

        * s/ms-w32.h: Guard MSC-specific #pragmas with an #ifdef.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: w32select.patch --]
[-- Type: text/x-patch, Size: 44037 bytes --]

Index: etc/NEWS
===================================================================
RCS file: /cvsroot/emacs/emacs/etc/NEWS,v
retrieving revision 1.1014
diff -u -p -r1.1014 NEWS
--- etc/NEWS	26 Aug 2004 01:47:55 -0000	1.1014
+++ etc/NEWS	30 Aug 2004 20:32:56 -0000
@@ -98,6 +98,9 @@ types any more.  Add -DUSE_LISP_UNION_TY
 \f
 * Changes in Emacs 21.4
 
+** On MS Windows NT/W2K/XP Emacs now uses Unicode for clipboard
+operations.
+
 ** global-whitespace-mode is a new alias for whitespace-global-mode.
 
 +++
Index: lisp/international/mule-cmds.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/international/mule-cmds.el,v
retrieving revision 1.257
diff -u -p -r1.257 mule-cmds.el
--- lisp/international/mule-cmds.el	12 Jun 2004 02:21:21 -0000	1.257
+++ lisp/international/mule-cmds.el	30 Aug 2004 20:32:57 -0000
@@ -2422,13 +2422,13 @@ See also `locale-charset-language-names'
 	  (prefer-coding-system coding-system)
 	  (setq locale-coding-system coding-system))))
 
-    ;; On Windows, override locale-coding-system, keyboard-coding-system,
-    ;; selection-coding-system with system codepage.
+    ;; On Windows, override locale-coding-system,
+    ;; keyboard-coding-system, with system codepage.  Note:
+    ;; selection-coding-system is already set in w32select.c.
     (when (boundp 'w32-ansi-code-page)
       (let ((code-page-coding (intern (format "cp%d" w32-ansi-code-page))))
 	(when (coding-system-p code-page-coding)
 	  (setq locale-coding-system code-page-coding)
-	  (set-selection-coding-system code-page-coding)
 	  (set-keyboard-coding-system code-page-coding)
 	  (set-terminal-coding-system code-page-coding))))
 
Index: man/msdog.texi
===================================================================
RCS file: /cvsroot/emacs/emacs/man/msdog.texi,v
retrieving revision 1.35
diff -u -p -r1.35 msdog.texi
--- man/msdog.texi	20 Jun 2004 14:45:40 -0000	1.35
+++ man/msdog.texi	30 Aug 2004 20:32:59 -0000
@@ -120,24 +120,30 @@ file:
 @end example
 
 @cindex Windows clipboard support
-  Emacs built for MS-DOS supports clipboard operations when it runs on
-Windows.  Commands that put text on the kill ring, or yank text from the
-ring, check the Windows clipboard first, just as Emacs does on the X
-Window System (@pxref{Mouse Commands}).  Only the primary selection and
-the cut buffer are supported by MS-DOS Emacs on Windows; the secondary
-selection always appears as empty.
-
-  Due to the way clipboard access is implemented by Windows, the
-length of text you can put into the clipboard is limited by the amount
-of free DOS memory that is available to Emacs.  Usually, up to 620KB of
-text can be put into the clipboard, but this limit depends on the system
-configuration and is lower if you run Emacs as a subprocess of
+  Emacs built for Windows supports clipboard operations.  Commands
+that put text on the kill ring, or yank text from the ring, check the
+Windows clipboard first, just as Emacs does on the X Window System
+(@pxref{Mouse Commands}).  Emacs uses Unicode for clipboard operations
+on Windows NT, Windows 2000 and Windows XP.  On Windows 98 and Windows
+Me, the clipboard operations use the system's 8-bit encoding.
+
+  Emacs built for MS-DOS also supports the Windows clipboard when it
+runs on Windows.  Operations on the primary selection and the cut
+buffer are translated to operations on the Windows clipboard by MS-DOS
+Emacs on Windows; the secondary selection always appears as empty.
+
+  Due to the way clipboard access is implemented on MS-DOS, the length
+of text you can put into the clipboard is limited by the amount of
+free DOS memory that is available to Emacs.  Usually, up to 620KB of
+text can be put into the clipboard, but this limit depends on the
+system configuration and is lower if you run Emacs as a subprocess of
 another program.  If the killed text does not fit, Emacs outputs a
 message saying so, and does not put the text into the clipboard.
 
-  Null characters also cannot be put into the Windows clipboard.  If the
-killed text includes null characters, Emacs does not put such text into
-the clipboard, and displays in the echo area a message to that effect.
+  Null characters also cannot be put into the Windows clipboard.  If
+the killed text includes null characters, Emacs on MS-DOS does not put
+such text into the clipboard, and displays in the echo area a message
+to that effect.
 
 @vindex dos-display-scancodes
   The variable @code{dos-display-scancodes}, when non-@code{nil},
Index: src/emacs.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/emacs.c,v
retrieving revision 1.342
diff -u -p -r1.342 emacs.c
--- src/emacs.c	24 Jun 2004 20:24:52 -0000	1.342
+++ src/emacs.c	30 Aug 2004 20:33:02 -0000
@@ -42,6 +42,8 @@ Boston, MA 02111-1307, USA.  */
 
 #ifdef WINDOWSNT
 #include <fcntl.h>
+#include <windows.h> /* just for w32.h */
+#include "w32.h"
 #endif
 
 #include "lisp.h"
@@ -1578,6 +1580,7 @@ main (argc, argv
 #ifdef HAVE_NTGUI
       globals_of_w32fns ();
       globals_of_w32menu ();
+      globals_of_w32select ();
 #endif  /* end #ifdef HAVE_NTGUI */
     }
 
Index: src/w32.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/w32.c,v
retrieving revision 1.88
diff -u -p -r1.88 w32.c
--- src/w32.c	17 May 2004 21:33:16 -0000	1.88
+++ src/w32.c	30 Aug 2004 20:33:02 -0000
@@ -3858,6 +3858,8 @@ term_ntproc ()
   /* shutdown the socket interface if necessary */
   term_winsock ();
 #endif
+
+  term_w32select ();
 }
 
 void
Index: src/w32.h
===================================================================
RCS file: /cvsroot/emacs/emacs/src/w32.h,v
retrieving revision 1.15
diff -u -p -r1.15 w32.h
--- src/w32.h	1 Sep 2003 15:45:57 -0000	1.15
+++ src/w32.h	30 Aug 2004 20:33:02 -0000
@@ -122,16 +122,18 @@ extern void reset_standard_handles (int 
 /* Return the string resource associated with KEY of type TYPE.  */
 extern LPBYTE w32_get_resource (char * key, LPDWORD type);
 
-extern void init_ntproc ();
-extern void term_ntproc ();
-extern void globals_of_w32 ();
-extern void syms_of_w32term ();
-extern void syms_of_w32fns ();
-extern void globals_of_w32fns ();
-extern void syms_of_w32select ();
-extern void syms_of_w32menu ();
-extern void globals_of_w32menu ();
-extern void syms_of_fontset ();
+extern void init_ntproc (void);
+extern void term_ntproc (void);
+extern void globals_of_w32 (void);
+extern void syms_of_w32term (void);
+extern void syms_of_w32fns (void);
+extern void globals_of_w32fns (void);
+extern void syms_of_w32select (void);
+extern void globals_of_w32select (void);
+extern void term_w32select (void);
+extern void syms_of_w32menu (void);
+extern void globals_of_w32menu (void);
+extern void syms_of_fontset (void);
 
 #endif /* EMACS_W32_H */
 
Index: src/w32select.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/w32select.c,v
retrieving revision 1.33
diff -u -p -r1.33 w32select.c
--- src/w32select.c	30 Jun 2004 23:30:39 -0000	1.33
+++ src/w32select.c	30 Aug 2004 20:33:02 -0000
@@ -1,5 +1,5 @@
 /* Selection processing for Emacs on the Microsoft W32 API.
-   Copyright (C) 1993, 1994 Free Software Foundation.
+   Copyright (C) 1993, 1994, 2004 Free Software Foundation.
 
 This file is part of GNU Emacs.
 
@@ -18,273 +18,673 @@ along with GNU Emacs; see the file COPYI
 the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
 Boston, MA 02111-1307, USA.  */
 
-/* Written by Kevin Gallo */
+/* Written by Kevin Gallo, Benjamin Riefenstahl */
 
 #include <config.h>
 #include "lisp.h"
 #include "w32term.h"	/* for all of the w32 includes */
-#include "dispextern.h"	/* frame.h seems to want this */
-#include "keyboard.h"
-#include "frame.h"	/* Need this to get the X window of selected_frame */
+#include "w32heap.h"	/* os_subtype */
 #include "blockinput.h"
-#include "buffer.h"
+#include "keyboard.h"	/* cmd_error_internal() */
 #include "charset.h"
 #include "coding.h"
 #include "composite.h"
 
+
+static HGLOBAL convert_to_handle_as_ascii (void);
+static HGLOBAL convert_to_handle_as_coded (Lisp_Object coding_system);
+static Lisp_Object render (Lisp_Object oformat);
+static Lisp_Object render_all (void);
+static void run_protected (Lisp_Object (*code) (), Lisp_Object arg);
+static Lisp_Object lisp_error_handler (Lisp_Object error);
+static LRESULT CALLBACK owner_callback (HWND win, UINT msg,
+					WPARAM wp, LPARAM lp);
+static HWND create_owner (void);
+
+static void setup_config (void);
+static BOOL WINAPI enum_locale_callback (/*const*/ char* loc_string);
+static UINT cp_from_locale (LCID lcid, UINT format);
+static Lisp_Object coding_from_cp (UINT codepage);
+
+
 Lisp_Object QCLIPBOARD;
 
 /* Coding system for communicating with other Windows programs via the
    clipboard.  */
 static Lisp_Object Vselection_coding_system;
 
-/* Coding system for the next communicating with other Windows programs.  */
+/* Coding system for the next communication with other Windows
+   programs.  */
 static Lisp_Object Vnext_selection_coding_system;
 
-/* Sequence number, used where possible to detect when we are pasting
-   our own text.  */
-static DWORD last_clipboard_sequence_number;
-extern ClipboardSequence_Proc clipboard_sequence_fn;
-
-/* The last text we put into the clipboard.  This is used when the OS
-   does not support sequence numbers (NT4, 95). It is undesirable to
-   use data put on the clipboard by Emacs because the clipboard data
-   could be MULEtilated by inappropriately chosen
-   (next-)selection-coding-system.  For this reason, we must store the
-   text *after* it was encoded/Unix-to-DOS-converted.  */
-static unsigned char *last_clipboard_text = NULL;
-static size_t clipboard_storage_size = 0;
-
-#if 0
-DEFUN ("w32-open-clipboard", Fw32_open_clipboard, Sw32_open_clipboard, 0, 1, 0,
-       doc: /* This opens the clipboard with the given frame pointer.  */)
-     (frame)
-     Lisp_Object frame;
+/* Internal pseudo-constants, initialized in globals_of_w32select()
+   based on current system parameters. */
+static LCID DEFAULT_LCID;
+static UINT ANSICP, OEMCP;
+static Lisp_Object QUNICODE, QANSICP, QOEMCP;
+
+/* A hidden window just for the clipboard management. */
+static HWND clipboard_owner;
+/* A flag to tell WM_DESTROYCLIPBOARD who is to blame this time (just
+   checking GetClipboardOwner() doesn't work, sadly). */
+static int modifying_clipboard = 0;
+
+/* Configured transfer parameters, based on the last inspection of
+   selection-coding-system.  */
+static Lisp_Object cfg_coding_system;
+static UINT cfg_codepage;
+static LCID cfg_lcid;
+static UINT cfg_clipboard_type;
+
+/* The current state for delayed rendering. */
+static Lisp_Object current_text;
+static Lisp_Object current_coding_system;
+static int current_requires_encoding, current_num_nls;
+static UINT current_clipboard_type;
+static LCID current_lcid;
+
+#if TRACE
+#define ONTRACE(stmt) stmt
+#else
+#define ONTRACE(stmt) /*stmt*/
+#endif
+
+
+/* This function assumes that there is no multibyte character in
+   current_text, so we can short-cut encoding.  */
+
+static HGLOBAL
+convert_to_handle_as_ascii (void)
 {
-  BOOL ok = FALSE;
+  HGLOBAL htext = NULL;
+  int nbytes;
+  int truelen;
+  unsigned char *src;
+  unsigned char *dst;
 
-  if (!NILP (frame))
-    CHECK_LIVE_FRAME (frame);
+  ONTRACE (fprintf (stderr, "convert_to_handle_as_ascii\n"));
 
-  BLOCK_INPUT;
+  nbytes = SBYTES (current_text) + 1;
+  src = SDATA (current_text);
 
-  ok = OpenClipboard ((!NILP (frame) && FRAME_W32_P (XFRAME (frame))) ? FRAME_W32_WINDOW (XFRAME (frame)) : NULL);
+  /* We need to add to the size the number of LF chars where we have
+     to insert CR chars (the standard CF_TEXT clipboard format uses
+     CRLF line endings, while Emacs uses just LF internally).  */
 
-  UNBLOCK_INPUT;
+  truelen = nbytes + current_num_nls;
+
+  if ((htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, truelen)) == NULL)
+    return NULL;
+
+  if ((dst = (unsigned char *) GlobalLock (htext)) == NULL)
+    {
+      GlobalFree (htext);
+      return NULL;
+    }
+
+  /* convert to CRLF line endings expected by clipboard */
+  while (1)
+    {
+      unsigned char *next;
+      /* copy next line or remaining bytes including '\0' */
+      next = _memccpy (dst, src, '\n', nbytes);
+      if (next)
+	{
+	  /* copied one line ending with '\n' */
+	  int copied = next - dst;
+	  nbytes -= copied;
+	  src += copied;
+	  /* insert '\r' before '\n' */
+	  next[-1] = '\r';
+	  next[0] = '\n';
+	  dst = next + 1;
+	}
+      else
+	/* copied remaining partial line -> now finished */
+	break;
+    }
 
-  return (ok ? frame : Qnil);
+  GlobalUnlock (htext);
+
+  return htext;
 }
 
-DEFUN ("w32-empty-clipboard", Fw32_empty_clipboard,
-       Sw32_empty_clipboard, 0, 0, 0,
-       doc: /* Empty the clipboard.
-Assigns ownership of the clipboard to the window which opened it.  */)
-     ()
+/* This function assumes that there are multibyte or NUL characters in
+   current_text, or that we need to construct Unicode.  It runs the
+   text through the encoding machinery.  */
+
+static HGLOBAL
+convert_to_handle_as_coded (Lisp_Object coding_system)
 {
-  BOOL ok = FALSE;
+  HGLOBAL htext = NULL, htext2;
+  int nbytes;
+  unsigned char *src;
+  unsigned char *dst = NULL;
+  int bufsize;
+  struct coding_system coding;
+  Lisp_Object string = Qnil;
+  struct gcpro gcpro1;
+
+  ONTRACE (fprintf (stderr, "convert_to_handle_as_coded: %s\n",	
+		    SDATA (SYMBOL_NAME (coding_system))));
+
+  /* FIXME: It looks like GC_MARK_STACK == GC_MAKE_GCPROS_NOOPS on
+     Windows (see src/lisp.h and nt/config.nt), so this is probably
+     unnecessary.  If it was necessary, we'd probably have to protect
+     all the symbols, too. */
+  GCPRO1 (string);
+
+  setup_coding_system (Fcheck_coding_system (coding_system), &coding);
+  coding.src_multibyte = 1;
+  coding.dst_multibyte = 0;
+  /* Need to set COMPOSITION_DISABLED, otherwise Emacs crashes in
+     encode_coding_iso2022 trying to dereference a null pointer.  */
+  coding.composing = COMPOSITION_DISABLED;
+  if (coding.type == coding_type_iso2022)
+    coding.flags |= CODING_FLAG_ISO_SAFE;
+  coding.mode |= CODING_MODE_LAST_BLOCK;
+  /* Force DOS line-ends. */
+  coding.eol_type = CODING_EOL_CRLF;
+
+  if (SYMBOLP (coding.pre_write_conversion)
+      && !NILP (Ffboundp (coding.pre_write_conversion)))
+    string = run_pre_post_conversion_on_str (current_text, &coding, 1);
+  else
+    string = current_text;
 
-  BLOCK_INPUT;
+  nbytes = SBYTES (string);
+  src = SDATA (string);
 
-  ok = EmptyClipboard ();
+  bufsize = encoding_buffer_size (&coding, nbytes) +2;
+  htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, bufsize);
 
-  UNBLOCK_INPUT;
+  if (htext != NULL)
+    dst = (unsigned char *) GlobalLock (htext);
+
+  if (dst != NULL)
+    {
+      encode_coding (&coding, src, dst, nbytes, bufsize-2);
+      /* Add the string terminator.  Add two NULs in case we are
+	 producing Unicode here.  */
+      dst[coding.produced] = dst[coding.produced+1] = '\0';
+    }
+
+  if (dst != NULL)
+    GlobalUnlock (htext);
+
+  if (htext != NULL)
+    {
+      /* Shrink data block to actual size.  */
+      htext2 = GlobalReAlloc (htext, coding.produced+2,
+			      GMEM_MOVEABLE | GMEM_DDESHARE);
+      if (htext2 != NULL) htext = htext2;
+    }
+
+  UNGCPRO;
+
+  return htext;
+}
+
+static Lisp_Object
+render (Lisp_Object oformat)
+{
+  HGLOBAL htext = NULL;
+  UINT format = XFASTINT (oformat);
+
+  ONTRACE (fprintf (stderr, "render\n"));
+
+  if (NILP (current_text))
+    return Qnil;
+
+  if (current_requires_encoding || format == CF_UNICODETEXT)
+    {
+      if (format == current_clipboard_type)
+	htext = convert_to_handle_as_coded (current_coding_system);
+      else
+	switch (format)
+	  {
+	  case CF_UNICODETEXT:
+	    htext = convert_to_handle_as_coded (QUNICODE);
+	    break;
+	  case CF_TEXT:
+	  case CF_OEMTEXT:
+	    {
+	      Lisp_Object cs;
+	      cs = coding_from_cp (cp_from_locale (current_lcid, format));
+	      htext = convert_to_handle_as_coded (cs);
+	      break;
+	    }
+	  }
+    }
+  else
+    htext = convert_to_handle_as_ascii ();
+
+  ONTRACE (fprintf (stderr, "render: htext = 0x%08X\n", (unsigned) htext));
 
-  return (ok ? Qt : Qnil);
+  if (htext != NULL)
+    SetClipboardData (format, htext);
+
+  return Qt;
 }
 
-DEFUN ("w32-close-clipboard", Fw32_close_clipboard,
-       Sw32_close_clipboard, 0, 0, 0,
-       doc: /* Close the clipboard.  */)
-     ()
+/* At the end of the program, we want to ensure that our clipboard
+   data survives us.  This code will do that.  */
+
+static Lisp_Object
+render_all (void)
 {
-  BOOL ok = FALSE;
+  ONTRACE (fprintf (stderr, "render_all\n"));
+
+  /* According to the docs we should not call OpenClipboard() here,
+     but testing on W2K and working code in other projects shows that
+     it is actually necessary.  */
+
+  OpenClipboard (NULL);
+
+  /* There is no usefull means to report errors here, there are none
+     expected anyway, and even if there were errors, they wouldn't do
+     any harm.  So we just go ahead and do what has to be done without
+     bothering with error handling.  */
+
+  ++modifying_clipboard;
+  EmptyClipboard ();
+  --modifying_clipboard;
+
+  /* For text formats that we don't render here, the OS can use its
+     own translation rules instead, so we don't really need to offer
+     everything.  To minimize memory consumption we cover three
+     possible situations based on our primary format as detected from
+     selection-coding-system (see setup_config()):
+
+     - Post CF_TEXT only.  Let Windows convert to CF_OEMTEXT and
+       Windows on NT or the application on 9x/Me convert to
+       CF_UNICODETEXT.
+
+     - Post CF_OEMTEXT only.  Similar automatic conversions happen as
+       for CF_TEXT.
+
+     - Post CF_UNICODETEXT + CF_TEXT.  9x itself ignores
+       CF_UNICODETEXT, even though some applications can still handle
+       it.
+
+       Note 1: We render the less capable CF_TEXT *before* the more
+       capable CF_UNICODETEXT, to prevent clobbering through automatic
+       conversions, just in case.
+
+       Note 2: We could check os_subtype here and only render the
+       additional CF_TEXT on 9x/Me.  But OTOH with
+       current_clipboard_type == CF_UNICODETEXT we don't involve the
+       automatic conversions anywhere else, so to get consistent
+       results, we probably don't want to rely on it here either.  */
+
+  if (current_clipboard_type == CF_UNICODETEXT)
+    render (make_number (CF_TEXT));
+  render (make_number (current_clipboard_type));
+
+  CloseClipboard ();
+
+  return Qnil;
+}
+
+static void
+run_protected (Lisp_Object (*code) (), Lisp_Object arg)
+{
+  extern int waiting_for_input;
+  int owfi;
 
   BLOCK_INPUT;
 
-  ok = CloseClipboard ();
+  /* Fsignal calls abort() if it sees that waiting_for_input set.  */
+  owfi = waiting_for_input;
+  waiting_for_input = 0;
+
+  internal_condition_case_1 (code, arg, Qt, lisp_error_handler);
+
+  waiting_for_input = owfi;
 
   UNBLOCK_INPUT;
+}
 
-  return (ok ? Qt : Qnil);
+static Lisp_Object
+lisp_error_handler (Lisp_Object error)
+{
+  Vsignaling_function = Qnil;
+  cmd_error_internal (error, "Error in delayed clipboard rendering: ");
+  Vinhibit_quit = Qt;
+  return Qt;
 }
 
-#endif
 
-DEFUN ("w32-set-clipboard-data", Fw32_set_clipboard_data,
-       Sw32_set_clipboard_data, 1, 2, 0,
-       doc: /* This sets the clipboard data to the given text.  */)
-    (string, frame)
-    Lisp_Object string, frame;
+static LRESULT CALLBACK
+owner_callback (HWND win, UINT msg, WPARAM wp, LPARAM lp)
 {
-  BOOL ok = TRUE;
-  HANDLE htext;
-  int nbytes;
-  int truelen, nlines = 0;
-  unsigned char *src;
-  unsigned char *dst;
+  switch (msg)
+    {
+    case WM_RENDERFORMAT:
+      ONTRACE (fprintf (stderr, "WM_RENDERFORMAT\n"));
+      run_protected (render, make_number (wp));
+      return 0;
+
+    case WM_RENDERALLFORMATS:
+      ONTRACE (fprintf (stderr, "WM_RENDERALLFORMATS\n"));
+      run_protected (render_all, Qnil);
+      return 0;
 
-  CHECK_STRING (string);
+    case WM_DESTROYCLIPBOARD:
+      if (!modifying_clipboard)
+	{
+	  ONTRACE (fprintf (stderr, "WM_DESTROYCLIPBOARD (other)\n"));
+	  current_text = Qnil;
+	  current_coding_system = Qnil;
+	}
+      else
+	{
+	  ONTRACE (fprintf (stderr, "WM_DESTROYCLIPBOARD (self)\n"));
+	}
+      return 0;
 
-  if (!NILP (frame))
-    CHECK_LIVE_FRAME (frame);
+    case WM_DESTROY:
+      if (win == clipboard_owner)
+	clipboard_owner = NULL;
+      break;
+    }
 
-  BLOCK_INPUT;
+  return DefWindowProc (win, msg, wp, lp);
+}
 
-  /* Include the terminating NULL character in the source of
-     conversion.  */
-  nbytes = SBYTES (string) + 1;
-  src = SDATA (string);
-  dst = src;
+static HWND
+create_owner (void)
+{
+  static const char CLASSNAME[] = "Emacs Clipboard";
+  WNDCLASS wc;
+  HWND hwnd;
+
+  memset (&wc, 0, sizeof (wc));
+  wc.lpszClassName = CLASSNAME;
+  wc.lpfnWndProc = owner_callback;
+  RegisterClass (&wc);
+
+  hwnd = CreateWindow (CLASSNAME, CLASSNAME, 0, 0, 0, 0, 0, NULL, NULL,
+		       NULL, NULL);
+
+  return hwnd;
+}
+
+/* Called on exit by term_ntproc() in w32.c */
 
-  /* We need to know how many lines there are, since we need CRLF line
-     termination for compatibility with other Windows Programs.
-     avoid using strchr because it recomputes the length every time */
-  while ((dst = memchr (dst, '\n', nbytes - (dst - src))) != NULL)
+void
+term_w32select (void)
+{
+  /* Make sure to trigger WM_RENDERALLFORMATS. */
+  if (clipboard_owner != NULL)
+    DestroyWindow (clipboard_owner);
+}
+
+static void
+setup_config (void)
+{
+  const char *coding_name;
+  const char *cp;
+  char *end;
+  int slen;
+  Lisp_Object active_coding_system;
+
+  CHECK_SYMBOL (Vselection_coding_system);
+
+  /* Check if we have it cached */
+  active_coding_system = NILP (Vnext_selection_coding_system) ?
+    Vselection_coding_system : Vnext_selection_coding_system;
+  if (!NILP (cfg_coding_system)
+      && EQ (cfg_coding_system, active_coding_system))
+    return;
+  cfg_coding_system = active_coding_system;
+  
+  /* Set some sensible fallbacks */
+  cfg_codepage = ANSICP;
+  cfg_lcid = LOCALE_NEUTRAL;
+  cfg_clipboard_type = CF_TEXT;
+
+  /* Interpret the coding system symbol name */
+  coding_name = SDATA (SYMBOL_NAME (active_coding_system));
+
+  /* "(.*-)?utf-16.*" -> CF_UNICODETEXT */
+  cp = strstr (coding_name, "utf-16");
+  if (cp != NULL && (cp == coding_name || cp[-1] == '-'))
     {
-      nlines++;
-      dst++;
+      cfg_clipboard_type = CF_UNICODETEXT;
+      return;
     }
 
-  {
-    /* Since we are now handling multilingual text, we must consider
-       encoding text for the clipboard.  */
-    int charset_info = find_charset_in_text (src, SCHARS (string),
-					     nbytes, NULL, Qnil);
+  /* "cp[0-9]+.*" or "windows-[0-9]+.*" -> CF_TEXT or CF_OEMTEXT */
+  slen = strlen (coding_name);
+  if (slen >= 4 && coding_name[0] == 'c' && coding_name[1] == 'p')
+    cp = coding_name + 2;
+  else if (slen >= 10 && memcmp (coding_name, "windows-", 8) == 0)
+    cp = coding_name + 8;
+  else
+    return;
+
+  end = (char*)cp;
+  cfg_codepage = strtol (cp, &end, 10);
+
+  /* Error return from strtol() or number of digits < 2 -> Restore the
+     default and drop it. */
+  if (cfg_codepage == 0 || (end-cp) < 2 )
+    {
+      cfg_codepage = ANSICP;
+      return;
+    }
 
-    if (charset_info == 0)
-      {
-	/* No multibyte character in OBJ.  We need not encode it.  */
+  /* Is it the currently active system default? */
+  if (cfg_codepage == ANSICP)
+    {
+      /* cfg_clipboard_type = CF_TEXT; */
+      return;
+    }
+  if (cfg_codepage == OEMCP)
+    {
+      cfg_clipboard_type = CF_OEMTEXT;
+      return;
+    }
 
-	/* Need to know final size after CR chars are inserted (the
-	   standard CF_TEXT clipboard format uses CRLF line endings,
-	   while Emacs uses just LF internally).  */
+  /* Else determine a suitable locale the hard way. */
+  EnumSystemLocales (enum_locale_callback, LCID_INSTALLED);
+}
 
-	truelen = nbytes + nlines;
+static BOOL WINAPI
+enum_locale_callback (/*const*/ char* loc_string)
+{
+  LCID lcid;
+  UINT codepage;
 
-	if ((htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, truelen)) == NULL)
-	  goto error;
+  lcid = strtoul (loc_string, NULL, 16);
 
-	if ((dst = (unsigned char *) GlobalLock (htext)) == NULL)
-	  goto error;
+  /* Is the wanted codepage the "ANSI" codepage for this locale? */
+  codepage = cp_from_locale (lcid, CF_TEXT);
+  if (codepage == cfg_codepage)
+    {
+      cfg_lcid = lcid;
+      cfg_clipboard_type = CF_TEXT;
+      return FALSE; /* Stop enumeration */
+    }
+  
+  /* Is the wanted codepage the OEM codepage for this locale? */
+  codepage = cp_from_locale (lcid, CF_OEMTEXT);
+  if (codepage == cfg_codepage)
+    {
+      cfg_lcid = lcid;
+      cfg_clipboard_type = CF_OEMTEXT;
+      return FALSE; /* Stop enumeration */
+    }
 
-	/* convert to CRLF line endings expected by clipboard */
-	while (1)
-	  {
-	    unsigned char *next;
-	    /* copy next line or remaining bytes including '\0' */
-	    next = _memccpy (dst, src, '\n', nbytes);
-	    if (next)
-	      {
-		/* copied one line ending with '\n' */
-		int copied = next - dst;
-		nbytes -= copied;
-		src += copied;
-		/* insert '\r' before '\n' */
-		next[-1] = '\r';
-		next[0] = '\n';
-		dst = next + 1;
-	      }
-	    else
-	      /* copied remaining partial line -> now finished */
-	      break;
-	  }
+  return TRUE; /* Continue enumeration */
+}
 
-	GlobalUnlock (htext);
+static UINT
+cp_from_locale (LCID lcid, UINT format)
+{
+  char buffer[20] = "";
+  UINT variant, cp;
 
-	Vlast_coding_system_used = Qraw_text;
-      }
-    else
-      {
-	/* We must encode contents of OBJ to the selection coding
-           system. */
-	int bufsize;
-	struct coding_system coding;
-	HANDLE htext2;
+  variant =
+    format == CF_TEXT ? LOCALE_IDEFAULTANSICODEPAGE : LOCALE_IDEFAULTCODEPAGE;
 
-	if (NILP (Vnext_selection_coding_system))
-	  Vnext_selection_coding_system = Vselection_coding_system;
-	setup_coding_system
-	  (Fcheck_coding_system (Vnext_selection_coding_system), &coding);
-	if (SYMBOLP (coding.pre_write_conversion)
-	    && !NILP (Ffboundp (coding.pre_write_conversion)))
-	  {
-	    string = run_pre_post_conversion_on_str (string, &coding, 1);
-	    src = SDATA (string);
-	    /* Include the terminating NULL character in the source of
-	       conversion.  */
-	    nbytes = SBYTES (string) + 1;
-	  }
-	coding.src_multibyte = 1;
-	coding.dst_multibyte = 0;
-	/* Need to set COMPOSITION_DISABLED, otherwise Emacs crashes in
-	   encode_coding_iso2022 trying to dereference a null pointer.  */
-	coding.composing = COMPOSITION_DISABLED;
-	if (coding.type == coding_type_iso2022)
-	  coding.flags |= CODING_FLAG_ISO_SAFE;
-	Vnext_selection_coding_system = Qnil;
-	coding.mode |= CODING_MODE_LAST_BLOCK;
-	bufsize = encoding_buffer_size (&coding, nbytes);
-	if ((htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, bufsize)) == NULL)
-	  goto error;
-	if ((dst = (unsigned char *) GlobalLock (htext)) == NULL)
-	  goto error;
-	encode_coding (&coding, src, dst, nbytes, bufsize);
-	Vlast_coding_system_used = coding.symbol;
+  GetLocaleInfo (lcid, variant, buffer, sizeof (buffer));
+  cp = strtoul (buffer, NULL, 10);
 
-	/* If clipboard sequence numbers are not supported, keep a copy for
-	   later comparison.  */
-	if (!clipboard_sequence_fn)
-	  {
-	    /* Stash away the data we are about to put into the
-	       clipboard, so we could later check inside
-	       Fw32_get_clipboard_data whether the clipboard still
-	       holds our data.  */
-	    if (clipboard_storage_size < coding.produced)
-	      {
-		clipboard_storage_size = coding.produced + 100;
-		last_clipboard_text = (char *) xrealloc (last_clipboard_text,
-							 clipboard_storage_size);
-	      }
-	    if (last_clipboard_text)
-	      memcpy (last_clipboard_text, dst, coding.produced);
-	  }
+  if (cp == CP_ACP)
+    return ANSICP;
+  else if (cp == CP_OEMCP)
+    return OEMCP;
+  else
+    return cp;
+}
 
-	GlobalUnlock (htext);
+static Lisp_Object
+coding_from_cp (UINT codepage)
+{
+  char buffer[30];
+  sprintf (buffer, "cp%d-dos", (int) codepage);
+  return intern (buffer);
+  /* We don't need to check that this coding system exists right here,
+     because that is done when the coding system is actually
+     instantiated, i.e. it is passed through Fcheck_coding_system()
+     there.  */
+}
 
-	/* Shrink data block to actual size.  */
-	htext2 = GlobalReAlloc (htext, coding.produced,
-                                GMEM_MOVEABLE | GMEM_DDESHARE);
-	if (htext2 != NULL) htext = htext2;
-      }
-  }
 
-  if (!OpenClipboard ((!NILP (frame) && FRAME_W32_P (XFRAME (frame))) ? FRAME_W32_WINDOW (XFRAME (frame)) : NULL))
+DEFUN ("w32-set-clipboard-data", Fw32_set_clipboard_data,
+       Sw32_set_clipboard_data, 1, 2, 0,
+       doc: /* This sets the clipboard data to the given text.  */)
+    (string, ignored)
+    Lisp_Object string, ignored;
+{
+  BOOL ok = TRUE;
+  HGLOBAL hlocale = NULL;
+  int nbytes;
+  unsigned char *src;
+  unsigned char *dst;
+  unsigned char *end;
+
+  (void) ignored;
+
+  CHECK_STRING (string);
+
+  setup_config ();
+
+  current_text = string;
+  current_coding_system = cfg_coding_system;
+  current_clipboard_type = cfg_clipboard_type;
+  current_lcid = cfg_lcid;
+  current_num_nls = 0;
+  current_requires_encoding = 0;
+  
+  BLOCK_INPUT;
+
+  /* Check for non-ASCII characters.  While we are at it, count the
+     number of LFs, so we know how many CRs we will have to add later
+     (just in the case where we can use our internal ASCII rendering,
+     see code and note in convert_to_handle_as_ascii() above).  */
+  nbytes = SBYTES (string);
+  src = SDATA (string);
+
+  for (dst = src, end = src+nbytes; dst < end; dst++)
+    {
+      if (*dst == '\n')
+	current_num_nls++;
+      else if (*dst >= 0x80 || *dst == 0)
+	{
+	  current_requires_encoding = 1;
+	  break;
+	}
+    }
+
+  if (!current_requires_encoding)
+    {
+      /* If all we have is ASCII we don't need to pretend we offer
+	 anything fancy. */
+      current_coding_system = Qraw_text;
+      current_clipboard_type = CF_TEXT;
+    }
+  else
+    {
+      /* If we have something non-ASCII we may want to set a locale.
+	 We always do that directly (non-delayed), as it's just a
+	 small bit and it simplifies the rest of the code. */
+      if (cfg_lcid != LOCALE_NEUTRAL && cfg_lcid != DEFAULT_LCID)
+	{
+	  hlocale = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE,
+				 sizeof (cfg_lcid));
+	  if (hlocale != NULL)
+	    {
+	      LCID * lcid_ptr;
+	      if ((lcid_ptr = (LCID *) GlobalLock (hlocale)) != NULL)
+		{
+		  *lcid_ptr = cfg_lcid;
+		  GlobalUnlock (hlocale);
+		}
+	    }
+	}
+    }
+
+  if (!OpenClipboard (clipboard_owner))
     goto error;
 
-  ok = EmptyClipboard () && SetClipboardData (CF_TEXT, htext);
+  ++modifying_clipboard;
+  ok = EmptyClipboard ();
+  --modifying_clipboard;
+
+  if (ok)
+    {
+      if (clipboard_owner == NULL)
+	{
+	  /* If for some reason we don't have a clipboard_owner, we
+	     just set the text format as chosen by the configuration
+	     and than forget about the whole thing.  */
+	  render (make_number (current_clipboard_type));
+	  current_text = Qnil;
+	  current_coding_system = Qnil;
+	}
+      else
+	{
+	  /* Advertise all supported formats so that whatever the
+	     requester chooses, only one encoding step needs to be
+	     made.  This is intentionally different from what we do in
+	     the WM_RENDERALLFORMATS handler.  */
+	  SetClipboardData (CF_UNICODETEXT, NULL);
+	  SetClipboardData (CF_TEXT, NULL);
+	  SetClipboardData (CF_OEMTEXT, NULL);
+	}
+
+      if (hlocale != NULL)
+	{
+	  ok = SetClipboardData (CF_LOCALE, hlocale) != NULL;
+	  if (!ok) EmptyClipboard ();
+	}
+    }
 
   CloseClipboard ();
 
-  /* Common sense says to read the sequence number inside the
-     OpenClipboard/ CloseClipboard block to avoid race conditions
-     where another app puts something on the clipboard straight after
-     us. But experience suggests that the sequence number from the
-     SetClipboardData is not allocated until we close the clipboard!
-     Since clipboard operations are normally user-driven, the race
-     condition is probably not going to really happen.  */
-  if (clipboard_sequence_fn)
-    last_clipboard_sequence_number = clipboard_sequence_fn ();
+  /* We haven't really "used" this coding system yet, and it's even
+     unclear if we ever will.  But this is a way to tell the upper
+     level what we *would* use under ideal circumstances.
+
+     We don't signal the actually used coding-system later when we
+     finally render, because that can happen at any time and we don't
+     want to disturb the "foreground" action. */
+  Vlast_coding_system_used = current_coding_system;
+
+  Vnext_selection_coding_system = Qnil;
 
   if (ok) goto done;
 
  error:
 
   ok = FALSE;
-  if (htext) GlobalFree (htext);
-  if (last_clipboard_text)
-    *last_clipboard_text = '\0';
+  if (hlocale) GlobalFree (hlocale);
 
-  last_clipboard_sequence_number = 0;
+  current_text = Qnil;
+  current_coding_system = Qnil;
 
  done:
   UNBLOCK_INPUT;
@@ -292,24 +692,50 @@ DEFUN ("w32-set-clipboard-data", Fw32_se
   return (ok ? string : Qnil);
 }
 
+
 DEFUN ("w32-get-clipboard-data", Fw32_get_clipboard_data,
        Sw32_get_clipboard_data, 0, 1, 0,
        doc: /* This gets the clipboard data in text format.  */)
-     (frame)
-     Lisp_Object frame;
+     (ignored)
+     Lisp_Object ignored;
 {
-  HANDLE htext;
+  HGLOBAL htext;
   Lisp_Object ret = Qnil;
+  UINT actual_clipboard_type;
+  int use_configured_coding_system = 1;
 
-  if (!NILP (frame))
-    CHECK_LIVE_FRAME (frame);
+  (void) ignored;
+
+  /* Don't pass our own text from the clipboard (which might be
+     troublesome if the killed text includes null characters).  */
+  if (!NILP (current_text))
+    return ret;
+
+  setup_config ();
+  actual_clipboard_type = cfg_clipboard_type;
 
   BLOCK_INPUT;
 
-  if (!OpenClipboard ((!NILP (frame) && FRAME_W32_P (XFRAME (frame))) ? FRAME_W32_WINDOW (XFRAME (frame)) : NULL))
+  if (!OpenClipboard (clipboard_owner))
     goto done;
 
-  if ((htext = GetClipboardData (CF_TEXT)) == NULL)
+  if ((htext = GetClipboardData (actual_clipboard_type)) == NULL)
+    {
+      /* If we want CF_UNICODETEXT but can't get it, the current
+	 coding system is useless.  OTOH we can still try and decode
+	 CF_TEXT based on the locale that the system gives us and that
+	 we get down below.  */
+      if (actual_clipboard_type == CF_UNICODETEXT)
+	{
+	  htext = GetClipboardData (CF_TEXT);
+	  if (htext != NULL)
+	    {
+	      actual_clipboard_type = CF_TEXT;
+	      use_configured_coding_system = 0;
+	    }
+	}
+    }
+  if (htext == NULL)
     goto closeclip;
 
   {
@@ -322,53 +748,107 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
     if ((src = (unsigned char *) GlobalLock (htext)) == NULL)
       goto closeclip;
 
-    nbytes = strlen (src);
-
-    /* If the text in clipboard is identical to what we put there
-       last time w32_set_clipboard_data was called, pretend there's no
-       data in the clipboard.  This is so we don't pass our own text
-       from the clipboard (which might be troublesome if the killed
-       text includes null characters).  */
-    if ((clipboard_sequence_fn
-	 && clipboard_sequence_fn () == last_clipboard_sequence_number)
-	|| (last_clipboard_text
-	    && clipboard_storage_size >= nbytes
-	    && memcmp(last_clipboard_text, src, nbytes) == 0))
-      goto closeclip;
+    /* If the clipboard data contains any non-ascii code, we need to
+       decode it with a coding system.
+       FIXME: Repeat the code for the Unicode case. */
+    if (actual_clipboard_type == CF_UNICODETEXT)
+      {
+	nbytes = lstrlenW ((WCHAR *)src) * 2;
+	require_decoding = 1;
+      }
+    else
+      {
+	int i;
 
-    {
-      /* If the clipboard data contains any non-ascii code, we
-	 need to decode it.  */
-      int i;
+	nbytes = strlen (src);
 
-      for (i = 0; i < nbytes; i++)
-	{
-	  if (src[i] >= 0x80)
-	    {
-	      require_decoding = 1;
-	      break;
-	    }
-	}
-    }
+	for (i = 0; i < nbytes; i++)
+	  {
+	    if (src[i] >= 0x80)
+	      {
+		require_decoding = 1;
+		break;
+	      }
+	  }
+      }
 
     if (require_decoding)
       {
 	int bufsize;
 	unsigned char *buf;
 	struct coding_system coding;
+	Lisp_Object coding_system = Qnil;
+	
+	/* `next-selection-coding-system' should override everything,
+	   even when the locale passed by the system disagrees.  The
+	   only exception is when `next-selection-coding-system'
+	   requested CF_UNICODETEXT and we couldn't get that. */
+	if (use_configured_coding_system
+	    && !NILP (Vnext_selection_coding_system))
+	    coding_system = Vnext_selection_coding_system;
+
+	/* If we have CF_TEXT or CF_OEMTEXT, we want to check out
+	   CF_LOCALE, too. */
+	else if (actual_clipboard_type != CF_UNICODETEXT)
+	  {
+	    HGLOBAL hlocale;
+	    LCID lcid = DEFAULT_LCID;
+	    UINT cp;
+
+	    /* Windows is documented to always generate CF_LOCALE info
+	       automatically, so the locale handle should be always
+	       present, but this is not always true on 9x ;-(.  */
+	    hlocale = GetClipboardData (CF_LOCALE);
+	    if (hlocale != NULL)
+	      {
+		const LCID * lcid_ptr;
+		lcid_ptr = (const LCID *) GlobalLock (hlocale);
+		if (lcid_ptr != NULL)
+		  {
+		    lcid = *lcid_ptr;
+		    GlobalUnlock (hlocale);
+		  }
+
+		/* 9x has garbage as the sort order (to be exact there
+		   is another instance of the language id in the upper
+		   word).  We don't care about sort order anyway, so
+		   we can just filter out the unneeded mis-information
+		   to avoid irritations. */
+		lcid = MAKELCID (LANGIDFROMLCID (lcid), SORT_DEFAULT);
+	      }
 
-	if (NILP (Vnext_selection_coding_system))
-	  Vnext_selection_coding_system = Vselection_coding_system;
-	setup_coding_system
-	  (Fcheck_coding_system (Vnext_selection_coding_system), &coding);
+	    /* If we are using fallback from CF_UNICODETEXT, we can't
+	       use the configured coding system.  Also we don't want
+	       to use it, if the system has supplied us with a locale
+	       and it is not just the system default. */
+	    if (!use_configured_coding_system || lcid != DEFAULT_LCID)
+	      {
+		cp = cp_from_locale (lcid, actual_clipboard_type);
+		/* If it's just our current standard setting anyway,
+		   use the coding system that the user has selected.
+		   Otherwise create a new spec to match the locale
+		   that was specified by the other side or the
+		   system.  */
+		if (!use_configured_coding_system || cp != cfg_codepage)
+		  coding_system = coding_from_cp (cp);
+	      }
+	  }
+
+	if (NILP (coding_system))
+	  coding_system = Vselection_coding_system;
+	Vnext_selection_coding_system = Qnil;
+
+	setup_coding_system (Fcheck_coding_system (coding_system), &coding);
 	coding.src_multibyte = 0;
 	coding.dst_multibyte = 1;
-	Vnext_selection_coding_system = Qnil;
 	coding.mode |= CODING_MODE_LAST_BLOCK;
 	/* We explicitely disable composition handling because
 	   selection data should not contain any composition
 	   sequence.  */
 	coding.composing = COMPOSITION_DISABLED;
+	/* Force DOS line-ends. */
+	coding.eol_type = CODING_EOL_CRLF;
+
 	bufsize = decoding_buffer_size (&coding, nbytes);
 	buf = (unsigned char *) xmalloc (bufsize);
 	decode_coding (&coding, src, buf, nbytes, bufsize);
@@ -382,10 +862,10 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
       }
     else
       {
-	/* Need to know final size after CR chars are removed because we
-	   can't change the string size manually, and doing an extra
-	   copy is silly.  Note that we only remove CR when it appears
-	   as part of CRLF.  */
+	/* Need to know final size after CR chars are removed because
+	   we can't change the string size manually, and doing an
+	   extra copy is silly.  We only remove CR when it appears as
+	   part of CRLF.  */
 
 	truelen = nbytes;
 	dst = src;
@@ -462,9 +942,14 @@ and t is the same as `SECONDARY'.  */)
 
       if (OpenClipboard (NULL))
 	{
-	  int format = 0;
-	  while (format = EnumClipboardFormats (format))
-	    if (format == CF_TEXT)
+	  UINT format = 0;
+	  setup_config ();
+	  while ((format = EnumClipboardFormats (format)))
+	    /* Check CF_TEXT in addition to cfg_clipboard_type,
+	       because we can fall back on that if CF_UNICODETEXT is
+	       not available.  Actually a check for CF_TEXT only
+	       should be enough.  */
+	    if (format == cfg_clipboard_type || format == CF_TEXT)
 	      {
 		val = Qt;
 		break;
@@ -476,24 +961,25 @@ and t is the same as `SECONDARY'.  */)
   return Qnil;
 }
 
+/* One-time init.  Called in the un-dumped Emacs, but not in the
+   dumped version. */
+
 void
 syms_of_w32select ()
 {
-#if 0
-  defsubr (&Sw32_open_clipboard);
-  defsubr (&Sw32_empty_clipboard);
-  defsubr (&Sw32_close_clipboard);
-#endif
   defsubr (&Sw32_set_clipboard_data);
   defsubr (&Sw32_get_clipboard_data);
   defsubr (&Sx_selection_exists_p);
 
   DEFVAR_LISP ("selection-coding-system", &Vselection_coding_system,
 	       doc: /* Coding system for communicating with other programs.
-When sending or receiving text via cut_buffer, selection, and clipboard,
-the text is encoded or decoded by this coding system.
-The default value is `iso-latin-1-dos'.  */);
-  Vselection_coding_system = intern ("iso-latin-1-dos");
+When sending or receiving text via cut_buffer, selection, and
+clipboard, the text is encoded or decoded by this coding system.
+The default value is the current system default encoding on 9x/Me and
+`utf-16le-dos' (Unicode) on NT/W2K/XP. */);  
+  /* The actual value is set dynamically in the dumped Emacs, see
+     below. */
+  Vselection_coding_system = Qnil;
 
   DEFVAR_LISP ("next-selection-coding-system", &Vnext_selection_coding_system,
 	       doc: /* Coding system for the next communication with other programs.
@@ -504,6 +990,41 @@ set to nil.  */);
   Vnext_selection_coding_system = Qnil;
 
   QCLIPBOARD = intern ("CLIPBOARD");	staticpro (&QCLIPBOARD);
+
+  cfg_coding_system = Qnil;     staticpro (&cfg_coding_system);
+  current_text = Qnil;		staticpro (&current_text);
+  current_coding_system = Qnil; staticpro (&current_coding_system);
+
+  QUNICODE = intern ("utf-16le-dos"); staticpro (&QUNICODE);
+  QANSICP = Qnil; staticpro (&QANSICP);
+  QOEMCP = Qnil;  staticpro (&QOEMCP);
+}
+
+/* One-time init.  Called in the dumped Emacs, but not in the
+   un-dumped version. */
+
+void
+globals_of_w32select ()
+{
+  DEFAULT_LCID = GetUserDefaultLCID ();
+  /* Drop the sort order from the LCID, so we can compare this with
+     CF_LOCALE objects that have the same fix on 9x.  */
+  DEFAULT_LCID = MAKELCID (LANGIDFROMLCID (DEFAULT_LCID), SORT_DEFAULT);
+
+  ANSICP = GetACP ();
+  OEMCP = GetOEMCP ();
+
+  QANSICP = coding_from_cp (ANSICP);
+  QOEMCP = coding_from_cp (OEMCP);
+
+  if (os_subtype == OS_NT)
+    Vselection_coding_system = QUNICODE;
+  else if (inhibit_window_system)
+    Vselection_coding_system = QOEMCP;
+  else
+    Vselection_coding_system = QANSICP;
+
+  clipboard_owner = create_owner ();
 }
 
 /* arch-tag: c96e9724-5eb1-4dad-be07-289f092fd2af
Index: src/s/ms-w32.h
===================================================================
RCS file: /cvsroot/emacs/emacs/src/s/ms-w32.h,v
retrieving revision 1.30
diff -u -p -r1.30 ms-w32.h
--- src/s/ms-w32.h	1 Sep 2003 15:45:58 -0000	1.30
+++ src/s/ms-w32.h	30 Aug 2004 20:33:02 -0000
@@ -477,8 +477,10 @@ extern char *get_emacs_configuration_opt
    must include config.h to pick up this pragma.  */
 
 /* Names must be < 8 bytes */
+#ifdef _MSC_VER
 #pragma data_seg("EMDATA")
 #pragma bss_seg("EMBSS")
+#endif
 
 /* #define FULL_DEBUG */
 /* #define EMACSDEBUG */

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Unicode support for the MS Windows clipboard [new patch]

[Eli Zaretskii]

> Cc: eliz@gnu.org,  emacs-devel@gnu.org,  monnier@iro.umontreal.ca,
> 	  handa@m17n.org,  jasonr@gnu.org
> From: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de>
> Date: Mon, 30 Aug 2004 22:47:46 +0200
> 
> Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:
> > I will fix the atexit() bit and see what I can do in terms of docs.
> 
> See attached patch. 
> 
> I have kept the documentation rather high-level, because users
> shouldn't need to do anything, the defaults are fine.  A more detailed
> documentation would have to go into the ELisp manual IMO.

I don't think the documentation of this should go into msdog.texi.
That section is for the MS-DOS port, and in the printed version it
goes into an appendix.  Windows users will not expect to find it
there.

I think we should leave msdog.texi alone and instead put the
Windows-related text into mule.texi (since the only different aspect
is the encoding/decoding issue).

Re: Unicode support for the MS Windows clipboard [new patch]

[Tak Ota]

I have confirmed both mingw build and MSVC build work fine in copying
and pasting unicode characters back and forth between emacs and
windoze application.  If there is no issues about this patch could
someone check it into CVS please.  Thanks.

-Tak


Mon, 30 Aug 2004 22:47:46 +0200: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> wrote:

> Hi all,
> 
> Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:
> > I will fix the atexit() bit and see what I can do in terms of docs.
> 
> See attached patch. 
> 
> I have kept the documentation rather high-level, because users
> shouldn't need to do anything, the defaults are fine.  A more detailed
> documentation would have to go into the ELisp manual IMO.
> 
> benny
> 
> 
> 2004-08-30  Benjamin Riefenstahl  <Benjamin.Riefenstahl@epost.de>
> 
> 	* w32select.c: Summary: Thorough rework to implement Unicode
>         clipboard operations and delayed rendering.
> 
>         Drop last_clipboard_text and related code, keep track of
>         ownership via clipboard_owner instead.  Drop old #if0
>         sections.
> 
> 	(DEFAULT_LCID, ANSICP, OEMCP, QUNICODE, QANSICP, QOEMCP)
> 	(clipboard_owner, modifying_clipboard, cfg_coding_system)
> 	(cfg_codepage, cfg_lcid, cfg_clipboard_type, current_text)
> 	(current_coding_system, current_requires_encoding)
> 	(current_num_nls, current_clipboard_type, current_lcid): New
> 	static variables.
> 
> 	(convert_to_handle_as_ascii, convert_to_handle_as_coded)
>         (render, render_all, run_protected, lisp_error_handler)
>         (owner_callback, create_owner, setup_config)
>         (enum_locale_callback, cp_from_locale, coding_from_cp): New
>         local functions.
> 
>         (term_w32select, globals_of_w32select): New global functions.
> 
> 	(Fw32_set_clipboard_data): Ignore parameter FRAME, use
> 	clipboard_owner instead.  Use delayed rendering and provide
> 	all text formats.  Provide CF_LOCALE if necessary.
> 
> 	(Fw32_get_clipboard_data): Handle CF_UNICODETEXT and
> 	CF_LOCALE.  Fall back to CF_TEXT, if CF_UNICODETEXT is not
> 	available.  Force DOS line-ends for decoding.
> 
> 	(Fx_selection_exists_p): Handle CF_UNICODETEXT.
> 
> 	(syms_of_w32select): Init and register new variables.
> 
>         * w32.h: Add prototypes for globals_of_w32select and
>         term_w32select.  Make the neighboring K&R declarations into
>         prototypes, too.
> 
>         * emacs.c: Include w32.h to get function prototypes.
>         (main): Call globals_of_w32select.
> 
>         * w32.c (term_ntproc): Call term_w32select. 
> 
>         * mule-cmds.el (set-locale-environment): Remove call to
>         set-selection-coding-system on Windows.
> 
>         * s/ms-w32.h: Guard MSC-specific #pragmas with an #ifdef.
> 
> 

Re: Unicode support for the MS Windows clipboard [new patch]

[Kim F. Storm]

Did anybody look at this patch ?
Should we install it?


Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> writes:

> 2004-08-30  Benjamin Riefenstahl  <Benjamin.Riefenstahl@epost.de>
>
> 	* w32select.c: Summary: Thorough rework to implement Unicode
>         clipboard operations and delayed rendering.
>
>         Drop last_clipboard_text and related code, keep track of
>         ownership via clipboard_owner instead.  Drop old #if0
>         sections.
>
> 	(DEFAULT_LCID, ANSICP, OEMCP, QUNICODE, QANSICP, QOEMCP)
> 	(clipboard_owner, modifying_clipboard, cfg_coding_system)
> 	(cfg_codepage, cfg_lcid, cfg_clipboard_type, current_text)
> 	(current_coding_system, current_requires_encoding)
> 	(current_num_nls, current_clipboard_type, current_lcid): New
> 	static variables.
>
> 	(convert_to_handle_as_ascii, convert_to_handle_as_coded)
>         (render, render_all, run_protected, lisp_error_handler)
>         (owner_callback, create_owner, setup_config)
>         (enum_locale_callback, cp_from_locale, coding_from_cp): New
>         local functions.
>
>         (term_w32select, globals_of_w32select): New global functions.
>
> 	(Fw32_set_clipboard_data): Ignore parameter FRAME, use
> 	clipboard_owner instead.  Use delayed rendering and provide
> 	all text formats.  Provide CF_LOCALE if necessary.
>
> 	(Fw32_get_clipboard_data): Handle CF_UNICODETEXT and
> 	CF_LOCALE.  Fall back to CF_TEXT, if CF_UNICODETEXT is not
> 	available.  Force DOS line-ends for decoding.
>
> 	(Fx_selection_exists_p): Handle CF_UNICODETEXT.
>
> 	(syms_of_w32select): Init and register new variables.
>
>         * w32.h: Add prototypes for globals_of_w32select and
>         term_w32select.  Make the neighboring K&R declarations into
>         prototypes, too.
>
>         * emacs.c: Include w32.h to get function prototypes.
>         (main): Call globals_of_w32select.
>
>         * w32.c (term_ntproc): Call term_w32select. 
>
>         * mule-cmds.el (set-locale-environment): Remove call to
>         set-selection-coding-system on Windows.
>
>         * s/ms-w32.h: Guard MSC-specific #pragmas with an #ifdef.
>

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Unicode support for the MS Windows clipboard [new patch]

[Jason Rumney]

Kim F. Storm wrote:

>Did anybody look at this patch ?
>Should we install it?
>  
>
Yes. It was reported that clipboard support works out of the box for the 
first time for Tak, so we should install it. I have been using the patch 
myself for about a month with no noticable problems.

I thought that Benjamin had access to CVS now, and would install it 
himself when he was ready. Was I wrong?
I also remember Eli making a comment about where the documentation 
should go.

Re: Unicode support for the MS Windows clipboard [new patch]

[Benjamin Riefenstahl]

Hi Jason,


Jason Rumney <jasonr@gnu.org> writes:
> [...] we should install it. [...]

Thanks for the testing andd the endorsement.

> I thought that Benjamin had access to CVS now, and would install it
> himself when he was ready. Was I wrong?

I don't have CVS access.

I also assume that I will have to sign a copyright assignment, as I
guess the change is not small enough to go in without that.

> I also remember Eli making a comment about where the documentation
> should go.

I have that on my to-do list for this weekend. 


benny

Re: Unicode support for the MS Windows clipboard [new patch]

[Richard Stallman]

I sent you (in another msg) the info on how to get the appropriate
papers.

Re: Unicode support for the MS Windows clipboard [new patch]

[Benjamin Riefenstahl]

Hi Richard,

Richard Stallman <rms@gnu.org> writes:
> I sent you (in another msg) the info on how to get the appropriate
> papers.

Thanks, I've followed that. 

benny

Re: Unicode support for the MS Windows clipboard [new patch]

[Benjamin Riefenstahl]

Hi Eli, Jason,


Sorry for taking so long to follow up. 

"Eli Zaretskii" <eliz@gnu.org> writes:
> I don't think the documentation of this should go into msdog.texi.
> That section is for the MS-DOS port, and in the printed version it
> goes into an appendix.  Windows users will not expect to find it
> there.

Well, as my defense, I seem to remember that folks on this list were
asked to look at the MS-DOS sections of the manual for
Windows-specific questions.  Also the index item "Windows clipboard
support" actually points to msdog.texi.  Maybe this should better be
"Windows clipboard support on MS-DOS", than.

> I think we should leave msdog.texi alone and instead put the
> Windows-related text into mule.texi (since the only different aspect
> is the encoding/decoding issue).

While set-selection-coding-system is discussed there shortly, neither
the default value nor any reason to customize it is actually
mentioned.  There may not be much reason to discuss actual
customization options for X11 but there is even less reason on
Windows.  So maybe we don't need any additional documentation here at
all?


I notice that I have left a few other questions un-answered before:


>> From: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de>
>> Date: Thu, 27 May 2004 11:45:42 +0200
>> - CF_OEMTEXT - If you want to cut-and-paste line drawing characters
>>   between Emacs and other console apps on 9x/Me this would be the type
>>   to use.

"Eli Zaretskii" <eliz@gnu.org> writes:
> Doesn't Unicode support line-drawing characters?

Yes, but lots of applications don't support Unicode and 9x/Me doesn't
provide automated conversion to the 8-bit encodings.


Jason Rumney <jasonr@gnu.org> writes:
> >>[Setting the default to Unicode on 9x/Me] There is no harm in doing
> >>it on 9x if we are also setting CF_TEXT manually.

Not sure what you mean with "setting CF_TEXT manually"?  If you mean
directly, non-delayed, the code doesn't usually do that (see below for
discussion of the one exception).

I think I have lost the context of that discussion a bit here.  I'll
just take the opportunity to discuss the actual behaviour of the code
with a bit more detail to clarify possible misunderstandings.

> Benjamin Riefenstahl wrote:
> >Neither the current default setting on 9x (matching cpXXXX), nor
> >utf-16le have any impact on normal operations, because they just
> >re-state the defaults, and they only apply to the respective
> >clipboard type.
> 
> I'm confused. If the default setting on 9x is cpXXXX, then
> CF_UNICODETEXT is not offered to the clipboard as an available
> format, or read from the clipboard, right?

No.  I implemented delayed rendering such that *all* possible formats
are always offered for rendering (CF_TEXT, CF_UNICODETEXT and
CF_OEMTEXT).  The pasting application gets to choose what it actually
prefers, and only when a paste is actually done, the data is
converted.

The customization of {next-}selection-coding-system only affects the
conversion of the one format of the three that matches the coding
system in question.  It doesn't affect which formats are offered.
Formats that do not match the current setting of the
{next-}selection-coding-system will use the Windows defaults.

The reason for that was the requirement not to introduce additional
config items.  Also, I imagine, that specific problems would be
limited to one of the three formats, actually I expect them only in
CF_TEXT or CF_OEMTEXT.  This would depend on the application that
Emacs is communicating with and its preferred format.

Implementing to offer all three formats was trivial once all cases
were implemented anyway so I didn't see a reason to limit this.

> In that case, Emacs is unable to transfer multilingual text between
> other Unicode aware programs such as MS Office, IE and
> Mozilla. Using utf-16le by default would allow this.

But if we *only* supported Unicode, Emacs would not be able to
transfer to non-Unicode applications at all on 9x/Me.


To expand on the "manual" setting remark above: There is only one
place where the system sets the clipboard directly, non-delayed.  That
is during Emacs exit (this may actually be were the discussion above
originated).  Naturally we can't use delayed rendering for whatever is
on the clipboard at this point any more, so I set the format that
matches the current selection-coding-system.  If that is
CF_UNICODETEXT, I also set CF_TEXT.  This is intended for 9x/Me, but
to keep the behaviour consistent, I also do this on NT-based systems.

How does the X11 delayed rendering code handle shut-down?


Jason Rumney <jasonr@gnu.org> writes:
> This might all become much simpler once the unicode branch is merged
> anyway.

You mention that hope in other posts, too.  I don't believe the
Unicode branch will make things any easier for this code.

Unicode on the inside of Emacs will at the most change the code to
actually do the encoding conversion, i.e. the small bits where the
coding-system is setup for encode_coding() and decode_coding().  But
the bulk of the code in w32select.c is about handling the Windows side
of things, and that will not change, unless the we were to drop
requirements, like e.g. 9x/Me support.


benny

Re: Unicode support for the MS Windows clipboard [new patch]

[Eli Zaretskii]

> Cc: Takaaki.Ota@am.sony.com,  emacs-devel@gnu.org,  jasonr@gnu.org
> From: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de>
> Date: Sun, 12 Sep 2004 21:50:14 +0200
> 
> "Eli Zaretskii" <eliz@gnu.org> writes:
> > I don't think the documentation of this should go into msdog.texi.
> > That section is for the MS-DOS port, and in the printed version it
> > goes into an appendix.  Windows users will not expect to find it
> > there.
> 
> Well, as my defense, I seem to remember that folks on this list were
> asked to look at the MS-DOS sections of the manual for
> Windows-specific questions.  Also the index item "Windows clipboard
> support" actually points to msdog.texi.  Maybe this should better be
> "Windows clipboard support on MS-DOS", than.

People were referred to the DOS section because in the past there was
no better place to look; historically, it so happened that the DOS
port was documented much better than the Windows port.  I believe this
is no longer the case.

The issues specific to the Windows port should IMHO be either in the
body of the manual or in a Windows-specific appendix.  This calls for
a volunteer; but once this is done, I promise to go through msdog.texi
and fix it so that it unequivocally refers to the MS-DOS port.

> > I think we should leave msdog.texi alone and instead put the
> > Windows-related text into mule.texi (since the only different aspect
> > is the encoding/decoding issue).
> 
> While set-selection-coding-system is discussed there shortly, neither
> the default value nor any reason to customize it is actually
> mentioned.  There may not be much reason to discuss actual
> customization options for X11 but there is even less reason on
> Windows.  So maybe we don't need any additional documentation here at
> all?

I think the clipboard handling is special enough to warrant some text
about it.  But that is just my HO; if the Windows experts think this
doesn't need to be documented, I won't object.

Re: Unicode support for the MS Windows clipboard [new patch]

[Benjamin Riefenstahl]

[-- Attachment #1: Type: text/plain, Size: 2449 bytes --]

Hi all,


Now that my papers are in order (;-)), here is a new version of the
much discussed patch to support the Unicode clipboard on Windows
(attached).

Notes:

- I didn't provide user-level documentation except a short mention in
  etc/NEWS.  The defaults should work fine and there should be less
  need to customize this than on X11.

- There is a short discussion of the use of the customization in
  comments in the code.  Something similar may be usefull in the ELisp
  manual, probably in "Window System Selections."


benny


2004-11-08  Benjamin Riefenstahl  <Benjamin.Riefenstahl@epost.de>

	* w32select.c: Summary: Thorough rework to implement Unicode
        clipboard operations and delayed rendering.

        Drop last_clipboard_text and related code, keep track of
        ownership via clipboard_owner instead.  Drop old #if0
        sections.

	(DEFAULT_LCID, ANSICP, OEMCP, QUNICODE, QANSICP, QOEMCP)
	(clipboard_owner, modifying_clipboard, cfg_coding_system)
	(cfg_codepage, cfg_lcid, cfg_clipboard_type, current_text)
	(current_coding_system, current_requires_encoding)
	(current_num_nls, current_clipboard_type, current_lcid): New
	static variables.

	(convert_to_handle_as_ascii, convert_to_handle_as_coded)
        (render, render_all, run_protected, lisp_error_handler)
        (owner_callback, create_owner, setup_config)
        (enum_locale_callback, cp_from_locale, coding_from_cp): New
        local functions.

        (term_w32select, globals_of_w32select): New global functions.

	(Fw32_set_clipboard_data): Ignore parameter FRAME, use
	clipboard_owner instead.  Use delayed rendering and provide
	all text formats.  Provide CF_LOCALE if necessary.

	(Fw32_get_clipboard_data): Handle CF_UNICODETEXT and
	CF_LOCALE.  Fall back to CF_TEXT, if CF_UNICODETEXT is not
	available.  Force DOS line-ends for decoding.

	(Fx_selection_exists_p): Handle CF_UNICODETEXT.

	(syms_of_w32select): Init and register new variables.

        * w32.h: Add prototypes for globals_of_w32select and
        term_w32select.  Make the neighboring K&R declarations into
        prototypes, too.

        * emacs.c: Include w32.h to get function prototypes.
        (main): Call globals_of_w32select.

        * w32.c (term_ntproc): Call term_w32select. 

        * mule-cmds.el (set-locale-environment): Remove call to
        set-selection-coding-system on Windows.

        * s/ms-w32.h: Guard MSC-specific #pragmas with an #ifdef.



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: w32select-4.patch --]
[-- Type: text/x-patch, Size: 44090 bytes --]

Index: etc/NEWS
===================================================================
RCS file: /cvsroot/emacs/emacs/etc/NEWS,v
retrieving revision 1.1062
diff -u -p -r1.1062 NEWS
--- etc/NEWS	6 Nov 2004 17:05:42 -0000	1.1062
+++ etc/NEWS	8 Nov 2004 17:25:29 -0000
@@ -98,6 +98,9 @@ types any more.  Add -DUSE_LISP_UNION_TY
 \f
 * Changes in Emacs 21.4
 
+** On MS Windows NT/W2K/XP Emacs now uses Unicode for clipboard
+operations.
+
 ** In Outline mode, hide-body no longer hides lines at the top
 of the file that precede the first header line.
 
Index: lisp/international/mule-cmds.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/international/mule-cmds.el,v
retrieving revision 1.258
diff -u -p -r1.258 mule-cmds.el
--- lisp/international/mule-cmds.el	4 Nov 2004 10:10:35 -0000	1.258
+++ lisp/international/mule-cmds.el	8 Nov 2004 17:25:30 -0000
@@ -2428,13 +2428,13 @@ See also `locale-charset-language-names'
 	  (prefer-coding-system coding-system)
 	  (setq locale-coding-system coding-system))))
 
-    ;; On Windows, override locale-coding-system, keyboard-coding-system,
-    ;; selection-coding-system with system codepage.
+    ;; On Windows, override locale-coding-system,
+    ;; keyboard-coding-system with system codepage.  Note:
+    ;; selection-coding-system is already set in w32select.c.
     (when (boundp 'w32-ansi-code-page)
       (let ((code-page-coding (intern (format "cp%d" w32-ansi-code-page))))
 	(when (coding-system-p code-page-coding)
 	  (setq locale-coding-system code-page-coding)
-	  (set-selection-coding-system code-page-coding)
 	  (set-keyboard-coding-system code-page-coding)
 	  (set-terminal-coding-system code-page-coding))))
 
Index: src/emacs.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/emacs.c,v
retrieving revision 1.346
diff -u -p -r1.346 emacs.c
--- src/emacs.c	7 Nov 2004 21:59:19 -0000	1.346
+++ src/emacs.c	8 Nov 2004 17:25:31 -0000
@@ -42,6 +42,8 @@ Boston, MA 02111-1307, USA.  */
 
 #ifdef WINDOWSNT
 #include <fcntl.h>
+#include <windows.h> /* just for w32.h */
+#include "w32.h"
 #endif
 
 #include "lisp.h"
@@ -1627,6 +1629,7 @@ main (argc, argv
 #ifdef HAVE_NTGUI
       globals_of_w32fns ();
       globals_of_w32menu ();
+      globals_of_w32select ();
 #endif  /* HAVE_NTGUI */
     }
 
Index: src/w32.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/w32.c,v
retrieving revision 1.89
diff -u -p -r1.89 w32.c
--- src/w32.c	19 Oct 2004 19:08:58 -0000	1.89
+++ src/w32.c	8 Nov 2004 17:25:32 -0000
@@ -3884,6 +3884,8 @@ term_ntproc ()
   /* shutdown the socket interface if necessary */
   term_winsock ();
 #endif
+
+  term_w32select ();
 }
 
 void
Index: src/w32.h
===================================================================
RCS file: /cvsroot/emacs/emacs/src/w32.h,v
retrieving revision 1.15
diff -u -p -r1.15 w32.h
--- src/w32.h	1 Sep 2003 15:45:57 -0000	1.15
+++ src/w32.h	8 Nov 2004 17:25:32 -0000
@@ -122,16 +122,18 @@ extern void reset_standard_handles (int 
 /* Return the string resource associated with KEY of type TYPE.  */
 extern LPBYTE w32_get_resource (char * key, LPDWORD type);
 
-extern void init_ntproc ();
-extern void term_ntproc ();
-extern void globals_of_w32 ();
-extern void syms_of_w32term ();
-extern void syms_of_w32fns ();
-extern void globals_of_w32fns ();
-extern void syms_of_w32select ();
-extern void syms_of_w32menu ();
-extern void globals_of_w32menu ();
-extern void syms_of_fontset ();
+extern void init_ntproc (void);
+extern void term_ntproc (void);
+extern void globals_of_w32 (void);
+extern void syms_of_w32term (void);
+extern void syms_of_w32fns (void);
+extern void globals_of_w32fns (void);
+extern void syms_of_w32select (void);
+extern void globals_of_w32select (void);
+extern void term_w32select (void);
+extern void syms_of_w32menu (void);
+extern void globals_of_w32menu (void);
+extern void syms_of_fontset (void);
 
 #endif /* EMACS_W32_H */
 
Index: src/w32select.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/w32select.c,v
retrieving revision 1.33
diff -u -p -r1.33 w32select.c
--- src/w32select.c	30 Jun 2004 23:30:39 -0000	1.33
+++ src/w32select.c	8 Nov 2004 17:25:32 -0000
@@ -1,5 +1,5 @@
 /* Selection processing for Emacs on the Microsoft W32 API.
-   Copyright (C) 1993, 1994 Free Software Foundation.
+   Copyright (C) 1993, 1994, 2004 Free Software Foundation.
 
 This file is part of GNU Emacs.
 
@@ -18,273 +18,744 @@ along with GNU Emacs; see the file COPYI
 the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
 Boston, MA 02111-1307, USA.  */
 
-/* Written by Kevin Gallo */
+/* Written by Kevin Gallo, Benjamin Riefenstahl */
 
+
+/*
+ * Notes on usage of selection-coding-system and
+ * next-selection-coding-system on MS Windows:
+ *
+ * The selection coding system variables apply only to the version of
+ * the clipboard data that is closest in type, i.e. when a 16-bit
+ * Unicode coding system is given, they apply to he Unicode clipboard
+ * (CF_UNICODETEXT), when a well-known console codepage is given, they
+ * apply to the console version of the clipboard data (CF_OEMTEXT),
+ * else they apply to the normal 8-bit text clipboard (CF_TEXT).
+ * 
+ * When pasting (getting data from the OS), the clipboard format that
+ * matches the {next-}selection-coding-system is retrieved.  If
+ * Unicode is requested, but not available, 8-bit text (CF_TEXT) is
+ * used.  In all other cases the OS will transparently convert
+ * formats, so no other fallback is needed.
+ *
+ * When copying or cutting (sending data to the OS), the data is
+ * announced and stored internally, but only actually rendered on
+ * request.  The requester determines the format provided.  The
+ * {next-}selection-coding-system is only used, when its corresponding
+ * clipboard type matches the type requested.
+ *
+ * Scenarios to use the facilities for customizing the selection
+ * coding system are:
+ * 
+ *   ;; Generally use KOI8-R instead of the russian MS codepage for
+ *   ;; the 8-bit clipboard.
+ *   (set-selection-coding-system 'koi8-r-dos)
+ * 
+ * Or
+ * 
+ *   ;; Create a special clipboard copy function that uses codepage
+ *   ;; 1253 (Greek) to copy Greek text to a specific non-Unicode
+ *   ;; application.
+ *   (defun greek-copy (beg end)
+ *     (interactive "r")
+ *     (set-next-selection-coding-system 'cp1253-dos)
+ *     (copy-region-as-kill beg end))
+ *   (global-set-key "\C-c\C-c" 'greek-copy)
+ */
+
+/*
+ * Ideas for further directions:
+ *
+ * The encoding and decoding routines could be moved to Lisp code
+ * similar to how xselect.c does it (using well-known routine names
+ * for the delayed rendering).  If the definition of which clipboard
+ * types should be supported is also moved to Lisp, functionality
+ * could be expanded to CF_HTML, CF_RTF and maybe other types.
+ */
+ 
 #include <config.h>
 #include "lisp.h"
 #include "w32term.h"	/* for all of the w32 includes */
-#include "dispextern.h"	/* frame.h seems to want this */
-#include "keyboard.h"
-#include "frame.h"	/* Need this to get the X window of selected_frame */
+#include "w32heap.h"	/* os_subtype */
 #include "blockinput.h"
-#include "buffer.h"
+#include "keyboard.h"	/* cmd_error_internal() */
 #include "charset.h"
 #include "coding.h"
 #include "composite.h"
 
+
+static HGLOBAL convert_to_handle_as_ascii (void);
+static HGLOBAL convert_to_handle_as_coded (Lisp_Object coding_system);
+static Lisp_Object render (Lisp_Object oformat);
+static Lisp_Object render_locale (void);
+static Lisp_Object render_all (void);
+static void run_protected (Lisp_Object (*code) (), Lisp_Object arg);
+static Lisp_Object lisp_error_handler (Lisp_Object error);
+static LRESULT CALLBACK owner_callback (HWND win, UINT msg,
+					WPARAM wp, LPARAM lp);
+static HWND create_owner (void);
+
+static void setup_config (void);
+static BOOL WINAPI enum_locale_callback (/*const*/ char* loc_string);
+static UINT cp_from_locale (LCID lcid, UINT format);
+static Lisp_Object coding_from_cp (UINT codepage);
+
+
+/* A remnant from X11: Symbol for the CLIPBORD selection type.  Other
+   selections are not used on Windows, so we don't need symbols for
+   PRIMARY and SECONDARY.  */
 Lisp_Object QCLIPBOARD;
 
-/* Coding system for communicating with other Windows programs via the
+/* Coding system for communicating with other programs via the
    clipboard.  */
 static Lisp_Object Vselection_coding_system;
 
-/* Coding system for the next communicating with other Windows programs.  */
+/* Coding system for the next communication with other programs.  */
 static Lisp_Object Vnext_selection_coding_system;
 
-/* Sequence number, used where possible to detect when we are pasting
-   our own text.  */
-static DWORD last_clipboard_sequence_number;
-extern ClipboardSequence_Proc clipboard_sequence_fn;
-
-/* The last text we put into the clipboard.  This is used when the OS
-   does not support sequence numbers (NT4, 95). It is undesirable to
-   use data put on the clipboard by Emacs because the clipboard data
-   could be MULEtilated by inappropriately chosen
-   (next-)selection-coding-system.  For this reason, we must store the
-   text *after* it was encoded/Unix-to-DOS-converted.  */
-static unsigned char *last_clipboard_text = NULL;
-static size_t clipboard_storage_size = 0;
-
-#if 0
-DEFUN ("w32-open-clipboard", Fw32_open_clipboard, Sw32_open_clipboard, 0, 1, 0,
-       doc: /* This opens the clipboard with the given frame pointer.  */)
-     (frame)
-     Lisp_Object frame;
+/* Internal pseudo-constants, initialized in globals_of_w32select()
+   based on current system parameters. */
+static LCID DEFAULT_LCID;
+static UINT ANSICP, OEMCP;
+static Lisp_Object QUNICODE, QANSICP, QOEMCP;
+
+/* A hidden window just for the clipboard management. */
+static HWND clipboard_owner;
+/* A flag to tell WM_DESTROYCLIPBOARD who is to blame this time (just
+   checking GetClipboardOwner() doesn't work, sadly). */
+static int modifying_clipboard = 0;
+
+/* Configured transfer parameters, based on the last inspection of
+   selection-coding-system.  */
+static Lisp_Object cfg_coding_system;
+static UINT cfg_codepage;
+static LCID cfg_lcid;
+static UINT cfg_clipboard_type;
+
+/* The current state for delayed rendering. */
+static Lisp_Object current_text;
+static Lisp_Object current_coding_system;
+static int current_requires_encoding, current_num_nls;
+static UINT current_clipboard_type;
+static LCID current_lcid;
+
+#if TRACE
+#define ONTRACE(stmt) stmt
+#else
+#define ONTRACE(stmt) /*stmt*/
+#endif
+
+
+/* This function assumes that there is no multibyte character in
+   current_text, so we can short-cut encoding.  */
+
+static HGLOBAL
+convert_to_handle_as_ascii (void)
 {
-  BOOL ok = FALSE;
+  HGLOBAL htext = NULL;
+  int nbytes;
+  int truelen;
+  unsigned char *src;
+  unsigned char *dst;
 
-  if (!NILP (frame))
-    CHECK_LIVE_FRAME (frame);
+  ONTRACE (fprintf (stderr, "convert_to_handle_as_ascii\n"));
 
-  BLOCK_INPUT;
+  nbytes = SBYTES (current_text) + 1;
+  src = SDATA (current_text);
 
-  ok = OpenClipboard ((!NILP (frame) && FRAME_W32_P (XFRAME (frame))) ? FRAME_W32_WINDOW (XFRAME (frame)) : NULL);
+  /* We need to add to the size the number of LF chars where we have
+     to insert CR chars (the standard CF_TEXT clipboard format uses
+     CRLF line endings, while Emacs uses just LF internally).  */
 
-  UNBLOCK_INPUT;
+  truelen = nbytes + current_num_nls;
+
+  if ((htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, truelen)) == NULL)
+    return NULL;
 
-  return (ok ? frame : Qnil);
+  if ((dst = (unsigned char *) GlobalLock (htext)) == NULL)
+    {
+      GlobalFree (htext);
+      return NULL;
+    }
+
+  /* convert to CRLF line endings expected by clipboard */
+  while (1)
+    {
+      unsigned char *next;
+      /* copy next line or remaining bytes including '\0' */
+      next = _memccpy (dst, src, '\n', nbytes);
+      if (next)
+	{
+	  /* copied one line ending with '\n' */
+	  int copied = next - dst;
+	  nbytes -= copied;
+	  src += copied;
+	  /* insert '\r' before '\n' */
+	  next[-1] = '\r';
+	  next[0] = '\n';
+	  dst = next + 1;
+	}
+      else
+	/* copied remaining partial line -> now finished */
+	break;
+    }
+
+  GlobalUnlock (htext);
+
+  return htext;
 }
 
-DEFUN ("w32-empty-clipboard", Fw32_empty_clipboard,
-       Sw32_empty_clipboard, 0, 0, 0,
-       doc: /* Empty the clipboard.
-Assigns ownership of the clipboard to the window which opened it.  */)
-     ()
+/* This function assumes that there are multibyte or NUL characters in
+   current_text, or that we need to construct Unicode.  It runs the
+   text through the encoding machinery.  */
+
+static HGLOBAL
+convert_to_handle_as_coded (Lisp_Object coding_system)
 {
-  BOOL ok = FALSE;
+  HGLOBAL htext = NULL, htext2;
+  int nbytes;
+  unsigned char *src;
+  unsigned char *dst = NULL;
+  int bufsize;
+  struct coding_system coding;
+  Lisp_Object string = Qnil;
+
+  ONTRACE (fprintf (stderr, "convert_to_handle_as_coded: %s\n",	
+		    SDATA (SYMBOL_NAME (coding_system))));
+
+  setup_coding_system (Fcheck_coding_system (coding_system), &coding);
+  coding.src_multibyte = 1;
+  coding.dst_multibyte = 0;
+  /* Need to set COMPOSITION_DISABLED, otherwise Emacs crashes in
+     encode_coding_iso2022 trying to dereference a null pointer.  */
+  coding.composing = COMPOSITION_DISABLED;
+  if (coding.type == coding_type_iso2022)
+    coding.flags |= CODING_FLAG_ISO_SAFE;
+  coding.mode |= CODING_MODE_LAST_BLOCK;
+  /* Force DOS line-ends. */
+  coding.eol_type = CODING_EOL_CRLF;
+
+  if (SYMBOLP (coding.pre_write_conversion)
+      && !NILP (Ffboundp (coding.pre_write_conversion)))
+    string = run_pre_post_conversion_on_str (current_text, &coding, 1);
+  else
+    string = current_text;
 
-  BLOCK_INPUT;
+  nbytes = SBYTES (string);
+  src = SDATA (string);
 
-  ok = EmptyClipboard ();
+  bufsize = encoding_buffer_size (&coding, nbytes) +2;
+  htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, bufsize);
 
-  UNBLOCK_INPUT;
+  if (htext != NULL)
+    dst = (unsigned char *) GlobalLock (htext);
+
+  if (dst != NULL)
+    {
+      encode_coding (&coding, src, dst, nbytes, bufsize-2);
+      /* Add the string terminator.  Add two NULs in case we are
+	 producing Unicode here.  */
+      dst[coding.produced] = dst[coding.produced+1] = '\0';
+    }
 
-  return (ok ? Qt : Qnil);
+  if (dst != NULL)
+    GlobalUnlock (htext);
+
+  if (htext != NULL)
+    {
+      /* Shrink data block to actual size.  */
+      htext2 = GlobalReAlloc (htext, coding.produced+2,
+			      GMEM_MOVEABLE | GMEM_DDESHARE);
+      if (htext2 != NULL) htext = htext2;
+    }
+
+  return htext;
 }
 
-DEFUN ("w32-close-clipboard", Fw32_close_clipboard,
-       Sw32_close_clipboard, 0, 0, 0,
-       doc: /* Close the clipboard.  */)
-     ()
+static Lisp_Object
+render (Lisp_Object oformat)
 {
-  BOOL ok = FALSE;
+  HGLOBAL htext = NULL;
+  UINT format = XFASTINT (oformat);
+
+  ONTRACE (fprintf (stderr, "render\n"));
+
+  if (NILP (current_text))
+    return Qnil;
+
+  if (current_requires_encoding || format == CF_UNICODETEXT)
+    {
+      if (format == current_clipboard_type)
+	htext = convert_to_handle_as_coded (current_coding_system);
+      else
+	switch (format)
+	  {
+	  case CF_UNICODETEXT:
+	    htext = convert_to_handle_as_coded (QUNICODE);
+	    break;
+	  case CF_TEXT:
+	  case CF_OEMTEXT:
+	    {
+	      Lisp_Object cs;
+	      cs = coding_from_cp (cp_from_locale (current_lcid, format));
+	      htext = convert_to_handle_as_coded (cs);
+	      break;
+	    }
+	  }
+    }
+  else
+    htext = convert_to_handle_as_ascii ();
+
+  ONTRACE (fprintf (stderr, "render: htext = 0x%08X\n", (unsigned) htext));
+
+  if (htext == NULL)
+    return Qnil;
+
+  if (SetClipboardData (format, htext) == NULL)
+    {
+      GlobalFree(htext);
+      return Qnil;
+    }
+
+  return Qt;
+}
+
+static Lisp_Object
+render_locale (void)
+{
+  HANDLE hlocale = NULL;
+  LCID * lcid_ptr;
+
+  ONTRACE (fprintf (stderr, "render_locale\n"));
+
+  if (current_lcid == LOCALE_NEUTRAL || current_lcid == DEFAULT_LCID)
+    return Qt;
+
+  hlocale = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, sizeof (current_lcid));
+  if (hlocale == NULL)
+    return Qnil;
+
+  if ((lcid_ptr = (LCID *) GlobalLock (hlocale)) == NULL)
+    {
+      GlobalFree(hlocale);
+      return Qnil;
+    }
+
+  *lcid_ptr = current_lcid;
+  GlobalUnlock (hlocale);
+
+  if (SetClipboardData (CF_LOCALE, hlocale) == NULL)
+    {
+      GlobalFree(hlocale);
+      return Qnil;
+    }
+
+  return Qt;
+}
+
+/* At the end of the program, we want to ensure that our clipboard
+   data survives us.  This code will do that.  */
+
+static Lisp_Object
+render_all (void)
+{
+  ONTRACE (fprintf (stderr, "render_all\n"));
+
+  /* According to the docs we should not call OpenClipboard() here,
+     but testing on W2K and working code in other projects shows that
+     it is actually necessary.  */
+
+  OpenClipboard (NULL);
+
+  /* There is no usefull means to report errors here, there are none
+     expected anyway, and even if there were errors, they wouldn't do
+     any harm.  So we just go ahead and do what has to be done without
+     bothering with error handling.  */
+
+  ++modifying_clipboard;
+  EmptyClipboard ();
+  --modifying_clipboard;
+
+  /* For text formats that we don't render here, the OS can use its
+     own translation rules instead, so we don't really need to offer
+     everything.  To minimize memory consumption we cover three
+     possible situations based on our primary format as detected from
+     selection-coding-system (see setup_config()):
+
+     - Post CF_TEXT only.  Let the OS convert to CF_OEMTEXT and the OS
+       (on NT) or the application (on 9x/Me) convert to
+       CF_UNICODETEXT.
+
+     - Post CF_OEMTEXT only.  Similar automatic conversions happen as
+       for CF_TEXT.
+
+     - Post CF_UNICODETEXT + CF_TEXT.  9x itself ignores
+       CF_UNICODETEXT, even though some applications can still handle
+       it.
+
+       Note 1: We render the less capable CF_TEXT *before* the more
+       capable CF_UNICODETEXT, to prevent clobbering through automatic
+       conversions, just in case.
+
+       Note 2: We could check os_subtype here and only render the
+       additional CF_TEXT on 9x/Me.  But OTOH with
+       current_clipboard_type == CF_UNICODETEXT we don't involve the
+       automatic conversions anywhere else, so to get consistent
+       results, we probably don't want to rely on it here either.  */
+
+  render_locale();
+
+  if (current_clipboard_type == CF_UNICODETEXT)
+    render (make_number (CF_TEXT));
+  render (make_number (current_clipboard_type));
+
+  CloseClipboard ();
+
+  return Qnil;
+}
+
+static void
+run_protected (Lisp_Object (*code) (), Lisp_Object arg)
+{
+  /* FIXME: This works but it doesn't feel right.  Too much fiddling
+     with global variables and calling strange looking functions.  Is
+     this really the right way to run Lisp callbacks?  */
+
+  extern int waiting_for_input;
+  int owfi;
 
   BLOCK_INPUT;
 
-  ok = CloseClipboard ();
+  /* Fsignal calls abort() if it sees that waiting_for_input is
+     set.  */
+  owfi = waiting_for_input;
+  waiting_for_input = 0;
+
+  internal_condition_case_1 (code, arg, Qt, lisp_error_handler);
+
+  waiting_for_input = owfi;
 
   UNBLOCK_INPUT;
+}
 
-  return (ok ? Qt : Qnil);
+static Lisp_Object
+lisp_error_handler (Lisp_Object error)
+{
+  Vsignaling_function = Qnil;
+  cmd_error_internal (error, "Error in delayed clipboard rendering: ");
+  Vinhibit_quit = Qt;
+  return Qt;
 }
 
-#endif
 
-DEFUN ("w32-set-clipboard-data", Fw32_set_clipboard_data,
-       Sw32_set_clipboard_data, 1, 2, 0,
-       doc: /* This sets the clipboard data to the given text.  */)
-    (string, frame)
-    Lisp_Object string, frame;
+static LRESULT CALLBACK
+owner_callback (HWND win, UINT msg, WPARAM wp, LPARAM lp)
 {
-  BOOL ok = TRUE;
-  HANDLE htext;
-  int nbytes;
-  int truelen, nlines = 0;
-  unsigned char *src;
-  unsigned char *dst;
+  switch (msg)
+    {
+    case WM_RENDERFORMAT:
+      ONTRACE (fprintf (stderr, "WM_RENDERFORMAT\n"));
+      run_protected (render, make_number (wp));
+      return 0;
+
+    case WM_RENDERALLFORMATS:
+      ONTRACE (fprintf (stderr, "WM_RENDERALLFORMATS\n"));
+      run_protected (render_all, Qnil);
+      return 0;
 
-  CHECK_STRING (string);
+    case WM_DESTROYCLIPBOARD:
+      if (!modifying_clipboard)
+	{
+	  ONTRACE (fprintf (stderr, "WM_DESTROYCLIPBOARD (other)\n"));
+	  current_text = Qnil;
+	  current_coding_system = Qnil;
+	}
+      else
+	{
+	  ONTRACE (fprintf (stderr, "WM_DESTROYCLIPBOARD (self)\n"));
+	}
+      return 0;
 
-  if (!NILP (frame))
-    CHECK_LIVE_FRAME (frame);
+    case WM_DESTROY:
+      if (win == clipboard_owner)
+	clipboard_owner = NULL;
+      break;
+    }
 
-  BLOCK_INPUT;
+  return DefWindowProc (win, msg, wp, lp);
+}
 
-  /* Include the terminating NULL character in the source of
-     conversion.  */
-  nbytes = SBYTES (string) + 1;
-  src = SDATA (string);
-  dst = src;
+static HWND
+create_owner (void)
+{
+  static const char CLASSNAME[] = "Emacs Clipboard";
+  WNDCLASS wc;
+
+  memset (&wc, 0, sizeof (wc));
+  wc.lpszClassName = CLASSNAME;
+  wc.lpfnWndProc = owner_callback;
+  RegisterClass (&wc);
+
+  return CreateWindow (CLASSNAME, CLASSNAME, 0, 0, 0, 0, 0, NULL, NULL,
+		       NULL, NULL);
+}
+
+/* Called on exit by term_ntproc() in w32.c */
+
+void
+term_w32select (void)
+{
+  /* This is needed to trigger WM_RENDERALLFORMATS. */
+  if (clipboard_owner != NULL)
+    DestroyWindow (clipboard_owner);
+}
 
-  /* We need to know how many lines there are, since we need CRLF line
-     termination for compatibility with other Windows Programs.
-     avoid using strchr because it recomputes the length every time */
-  while ((dst = memchr (dst, '\n', nbytes - (dst - src))) != NULL)
+static void
+setup_config (void)
+{
+  const char *coding_name;
+  const char *cp;
+  char *end;
+  int slen;
+  Lisp_Object new_coding_system;
+
+  CHECK_SYMBOL (Vselection_coding_system);
+
+  /* Check if we have it cached */
+  new_coding_system = NILP (Vnext_selection_coding_system) ?
+    Vselection_coding_system : Vnext_selection_coding_system;
+  if (!NILP (cfg_coding_system)
+      && EQ (cfg_coding_system, new_coding_system))
+    return;
+  cfg_coding_system = new_coding_system;
+  
+  /* Set some sensible fallbacks */
+  cfg_codepage = ANSICP;
+  cfg_lcid = LOCALE_NEUTRAL;
+  cfg_clipboard_type = CF_TEXT;
+
+  /* Interpret the coding system symbol name */
+  coding_name = SDATA (SYMBOL_NAME (cfg_coding_system));
+
+  /* "(.*-)?utf-16.*" -> CF_UNICODETEXT */
+  cp = strstr (coding_name, "utf-16");
+  if (cp != NULL && (cp == coding_name || cp[-1] == '-'))
     {
-      nlines++;
-      dst++;
+      cfg_clipboard_type = CF_UNICODETEXT;
+      return;
     }
 
-  {
-    /* Since we are now handling multilingual text, we must consider
-       encoding text for the clipboard.  */
-    int charset_info = find_charset_in_text (src, SCHARS (string),
-					     nbytes, NULL, Qnil);
+  /* "cp[0-9]+.*" or "windows-[0-9]+.*" -> CF_TEXT or CF_OEMTEXT */
+  slen = strlen (coding_name);
+  if (slen >= 4 && coding_name[0] == 'c' && coding_name[1] == 'p')
+    cp = coding_name + 2;
+  else if (slen >= 10 && memcmp (coding_name, "windows-", 8) == 0)
+    cp = coding_name + 8;
+  else
+    return;
+
+  end = (char*)cp;
+  cfg_codepage = strtol (cp, &end, 10);
+
+  /* Error return from strtol() or number of digits < 2 -> Restore the
+     default and drop it. */
+  if (cfg_codepage == 0 || (end-cp) < 2 )
+    {
+      cfg_codepage = ANSICP;
+      return;
+    }
 
-    if (charset_info == 0)
-      {
-	/* No multibyte character in OBJ.  We need not encode it.  */
+  /* Is it the currently active system default? */
+  if (cfg_codepage == ANSICP)
+    {
+      /* cfg_clipboard_type = CF_TEXT; */
+      return;
+    }
+  if (cfg_codepage == OEMCP)
+    {
+      cfg_clipboard_type = CF_OEMTEXT;
+      return;
+    }
+
+  /* Else determine a suitable locale the hard way. */
+  EnumSystemLocales (enum_locale_callback, LCID_INSTALLED);
+}
+
+static BOOL WINAPI
+enum_locale_callback (/*const*/ char* loc_string)
+{
+  LCID lcid;
+  UINT codepage;
 
-	/* Need to know final size after CR chars are inserted (the
-	   standard CF_TEXT clipboard format uses CRLF line endings,
-	   while Emacs uses just LF internally).  */
+  lcid = strtoul (loc_string, NULL, 16);
 
-	truelen = nbytes + nlines;
+  /* Is the wanted codepage the "ANSI" codepage for this locale? */
+  codepage = cp_from_locale (lcid, CF_TEXT);
+  if (codepage == cfg_codepage)
+    {
+      cfg_lcid = lcid;
+      cfg_clipboard_type = CF_TEXT;
+      return FALSE; /* Stop enumeration */
+    }
+  
+  /* Is the wanted codepage the OEM codepage for this locale? */
+  codepage = cp_from_locale (lcid, CF_OEMTEXT);
+  if (codepage == cfg_codepage)
+    {
+      cfg_lcid = lcid;
+      cfg_clipboard_type = CF_OEMTEXT;
+      return FALSE; /* Stop enumeration */
+    }
 
-	if ((htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, truelen)) == NULL)
-	  goto error;
+  return TRUE; /* Continue enumeration */
+}
 
-	if ((dst = (unsigned char *) GlobalLock (htext)) == NULL)
-	  goto error;
+static UINT
+cp_from_locale (LCID lcid, UINT format)
+{
+  char buffer[20] = "";
+  UINT variant, cp;
 
-	/* convert to CRLF line endings expected by clipboard */
-	while (1)
-	  {
-	    unsigned char *next;
-	    /* copy next line or remaining bytes including '\0' */
-	    next = _memccpy (dst, src, '\n', nbytes);
-	    if (next)
-	      {
-		/* copied one line ending with '\n' */
-		int copied = next - dst;
-		nbytes -= copied;
-		src += copied;
-		/* insert '\r' before '\n' */
-		next[-1] = '\r';
-		next[0] = '\n';
-		dst = next + 1;
-	      }
-	    else
-	      /* copied remaining partial line -> now finished */
-	      break;
-	  }
+  variant =
+    format == CF_TEXT ? LOCALE_IDEFAULTANSICODEPAGE : LOCALE_IDEFAULTCODEPAGE;
 
-	GlobalUnlock (htext);
+  GetLocaleInfo (lcid, variant, buffer, sizeof (buffer));
+  cp = strtoul (buffer, NULL, 10);
 
-	Vlast_coding_system_used = Qraw_text;
-      }
-    else
-      {
-	/* We must encode contents of OBJ to the selection coding
-           system. */
-	int bufsize;
-	struct coding_system coding;
-	HANDLE htext2;
+  if (cp == CP_ACP)
+    return ANSICP;
+  else if (cp == CP_OEMCP)
+    return OEMCP;
+  else
+    return cp;
+}
 
-	if (NILP (Vnext_selection_coding_system))
-	  Vnext_selection_coding_system = Vselection_coding_system;
-	setup_coding_system
-	  (Fcheck_coding_system (Vnext_selection_coding_system), &coding);
-	if (SYMBOLP (coding.pre_write_conversion)
-	    && !NILP (Ffboundp (coding.pre_write_conversion)))
-	  {
-	    string = run_pre_post_conversion_on_str (string, &coding, 1);
-	    src = SDATA (string);
-	    /* Include the terminating NULL character in the source of
-	       conversion.  */
-	    nbytes = SBYTES (string) + 1;
-	  }
-	coding.src_multibyte = 1;
-	coding.dst_multibyte = 0;
-	/* Need to set COMPOSITION_DISABLED, otherwise Emacs crashes in
-	   encode_coding_iso2022 trying to dereference a null pointer.  */
-	coding.composing = COMPOSITION_DISABLED;
-	if (coding.type == coding_type_iso2022)
-	  coding.flags |= CODING_FLAG_ISO_SAFE;
-	Vnext_selection_coding_system = Qnil;
-	coding.mode |= CODING_MODE_LAST_BLOCK;
-	bufsize = encoding_buffer_size (&coding, nbytes);
-	if ((htext = GlobalAlloc (GMEM_MOVEABLE | GMEM_DDESHARE, bufsize)) == NULL)
-	  goto error;
-	if ((dst = (unsigned char *) GlobalLock (htext)) == NULL)
-	  goto error;
-	encode_coding (&coding, src, dst, nbytes, bufsize);
-	Vlast_coding_system_used = coding.symbol;
+static Lisp_Object
+coding_from_cp (UINT codepage)
+{
+  char buffer[30];
+  sprintf (buffer, "cp%d-dos", (int) codepage);
+  return intern (buffer);
+  /* We don't need to check that this coding system exists right here,
+     because that is done when the coding system is actually
+     instantiated, i.e. it is passed through Fcheck_coding_system()
+     there.  */
+}
 
-	/* If clipboard sequence numbers are not supported, keep a copy for
-	   later comparison.  */
-	if (!clipboard_sequence_fn)
-	  {
-	    /* Stash away the data we are about to put into the
-	       clipboard, so we could later check inside
-	       Fw32_get_clipboard_data whether the clipboard still
-	       holds our data.  */
-	    if (clipboard_storage_size < coding.produced)
-	      {
-		clipboard_storage_size = coding.produced + 100;
-		last_clipboard_text = (char *) xrealloc (last_clipboard_text,
-							 clipboard_storage_size);
-	      }
-	    if (last_clipboard_text)
-	      memcpy (last_clipboard_text, dst, coding.produced);
-	  }
 
-	GlobalUnlock (htext);
+DEFUN ("w32-set-clipboard-data", Fw32_set_clipboard_data,
+       Sw32_set_clipboard_data, 1, 2, 0,
+       doc: /* This sets the clipboard data to the given text.  */)
+    (string, ignored)
+    Lisp_Object string, ignored;
+{
+  BOOL ok = TRUE;
+  int nbytes;
+  unsigned char *src;
+  unsigned char *dst;
+  unsigned char *end;
 
-	/* Shrink data block to actual size.  */
-	htext2 = GlobalReAlloc (htext, coding.produced,
-                                GMEM_MOVEABLE | GMEM_DDESHARE);
-	if (htext2 != NULL) htext = htext2;
-      }
-  }
+  /* This parameter used to be the current frame, but we don't use
+     that any more. */
+  (void) ignored;
+
+  CHECK_STRING (string);
+
+  setup_config ();
+
+  current_text = string;
+  current_coding_system = cfg_coding_system;
+  current_clipboard_type = cfg_clipboard_type;
+  current_lcid = cfg_lcid;
+  current_num_nls = 0;
+  current_requires_encoding = 0;
+  
+  BLOCK_INPUT;
+
+  /* Check for non-ASCII characters.  While we are at it, count the
+     number of LFs, so we know how many CRs we will have to add later
+     (just in the case where we can use our internal ASCII rendering,
+     see code and comment in convert_to_handle_as_ascii() above).  */
+  nbytes = SBYTES (string);
+  src = SDATA (string);
 
-  if (!OpenClipboard ((!NILP (frame) && FRAME_W32_P (XFRAME (frame))) ? FRAME_W32_WINDOW (XFRAME (frame)) : NULL))
+  for (dst = src, end = src+nbytes; dst < end; dst++)
+    {
+      if (*dst == '\n')
+	current_num_nls++;
+      else if (*dst >= 0x80 || *dst == 0)
+	{
+	  current_requires_encoding = 1;
+	  break;
+	}
+    }
+
+  if (!current_requires_encoding)
+    {
+      /* If all we have is ASCII we don't need to pretend we offer
+	 anything fancy. */
+      current_coding_system = Qraw_text;
+      current_clipboard_type = CF_TEXT;
+      current_lcid = LOCALE_NEUTRAL;
+    }
+
+  if (!OpenClipboard (clipboard_owner))
     goto error;
 
-  ok = EmptyClipboard () && SetClipboardData (CF_TEXT, htext);
+  ++modifying_clipboard;
+  ok = EmptyClipboard ();
+  --modifying_clipboard;
+
+  /* If we have something non-ASCII we may want to set a locale.  We
+     do that directly (non-delayed), as it's just a small bit.  */
+  if (ok)
+    ok = !NILP(render_locale());
+
+  if (ok)
+    {
+      if (clipboard_owner == NULL)
+	{
+	  /* If for some reason we don't have a clipboard_owner, we
+	     just set the text format as chosen by the configuration
+	     and than forget about the whole thing.  */
+	  ok = !NILP(render (make_number (current_clipboard_type)));
+	  current_text = Qnil;
+	  current_coding_system = Qnil;
+	}
+      else
+	{
+	  /* Advertise all supported formats so that whatever the
+	     requester chooses, only one encoding step needs to be
+	     made.  This is intentionally different from what we do in
+	     the handler for WM_RENDERALLFORMATS.  */
+	  SetClipboardData (CF_UNICODETEXT, NULL);
+	  SetClipboardData (CF_TEXT, NULL);
+	  SetClipboardData (CF_OEMTEXT, NULL);
+	}
+    }
 
   CloseClipboard ();
 
-  /* Common sense says to read the sequence number inside the
-     OpenClipboard/ CloseClipboard block to avoid race conditions
-     where another app puts something on the clipboard straight after
-     us. But experience suggests that the sequence number from the
-     SetClipboardData is not allocated until we close the clipboard!
-     Since clipboard operations are normally user-driven, the race
-     condition is probably not going to really happen.  */
-  if (clipboard_sequence_fn)
-    last_clipboard_sequence_number = clipboard_sequence_fn ();
+  /* With delayed rendering we haven't really "used" this coding
+     system yet, and it's even unclear if we ever will.  But this is a
+     way to tell the upper level what we *would* use under ideal
+     circumstances.
+
+     We don't signal the actually used coding-system later when we
+     finally render, because that can happen at any time and we don't
+     want to disturb the "foreground" action. */
+  if (ok)
+    Vlast_coding_system_used = current_coding_system;
+
+  Vnext_selection_coding_system = Qnil;
 
   if (ok) goto done;
 
  error:
 
   ok = FALSE;
-  if (htext) GlobalFree (htext);
-  if (last_clipboard_text)
-    *last_clipboard_text = '\0';
-
-  last_clipboard_sequence_number = 0;
+  current_text = Qnil;
+  current_coding_system = Qnil;
 
  done:
   UNBLOCK_INPUT;
@@ -292,24 +763,52 @@ DEFUN ("w32-set-clipboard-data", Fw32_se
   return (ok ? string : Qnil);
 }
 
+
 DEFUN ("w32-get-clipboard-data", Fw32_get_clipboard_data,
        Sw32_get_clipboard_data, 0, 1, 0,
        doc: /* This gets the clipboard data in text format.  */)
-     (frame)
-     Lisp_Object frame;
+     (ignored)
+     Lisp_Object ignored;
 {
-  HANDLE htext;
+  HGLOBAL htext;
   Lisp_Object ret = Qnil;
+  UINT actual_clipboard_type;
+  int use_configured_coding_system = 1;
 
-  if (!NILP (frame))
-    CHECK_LIVE_FRAME (frame);
+  /* This parameter used to be the current frame, but we don't use
+     that any more. */
+  (void) ignored;
+
+  /* Don't pass our own text from the clipboard (which might be
+     troublesome if the killed text includes null characters).  */
+  if (!NILP (current_text))
+    return ret;
+
+  setup_config ();
+  actual_clipboard_type = cfg_clipboard_type;
 
   BLOCK_INPUT;
 
-  if (!OpenClipboard ((!NILP (frame) && FRAME_W32_P (XFRAME (frame))) ? FRAME_W32_WINDOW (XFRAME (frame)) : NULL))
+  if (!OpenClipboard (clipboard_owner))
     goto done;
 
-  if ((htext = GetClipboardData (CF_TEXT)) == NULL)
+  if ((htext = GetClipboardData (actual_clipboard_type)) == NULL)
+    {
+      /* If we want CF_UNICODETEXT but can't get it, the current
+	 coding system is useless.  OTOH we can still try and decode
+	 CF_TEXT based on the locale that the system gives us and that
+	 we get down below.  */
+      if (actual_clipboard_type == CF_UNICODETEXT)
+	{
+	  htext = GetClipboardData (CF_TEXT);
+	  if (htext != NULL)
+	    {
+	      actual_clipboard_type = CF_TEXT;
+	      use_configured_coding_system = 0;
+	    }
+	}
+    }
+  if (htext == NULL)
     goto closeclip;
 
   {
@@ -322,53 +821,107 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
     if ((src = (unsigned char *) GlobalLock (htext)) == NULL)
       goto closeclip;
 
-    nbytes = strlen (src);
-
-    /* If the text in clipboard is identical to what we put there
-       last time w32_set_clipboard_data was called, pretend there's no
-       data in the clipboard.  This is so we don't pass our own text
-       from the clipboard (which might be troublesome if the killed
-       text includes null characters).  */
-    if ((clipboard_sequence_fn
-	 && clipboard_sequence_fn () == last_clipboard_sequence_number)
-	|| (last_clipboard_text
-	    && clipboard_storage_size >= nbytes
-	    && memcmp(last_clipboard_text, src, nbytes) == 0))
-      goto closeclip;
+    /* If the clipboard data contains any non-ascii code, we need to
+       decode it with a coding system.  */
+    if (actual_clipboard_type == CF_UNICODETEXT)
+      {
+	nbytes = lstrlenW ((WCHAR *)src) * 2;
+	require_decoding = 1;
+      }
+    else
+      {
+	int i;
 
-    {
-      /* If the clipboard data contains any non-ascii code, we
-	 need to decode it.  */
-      int i;
+	nbytes = strlen (src);
 
-      for (i = 0; i < nbytes; i++)
-	{
-	  if (src[i] >= 0x80)
-	    {
-	      require_decoding = 1;
-	      break;
-	    }
-	}
-    }
+	for (i = 0; i < nbytes; i++)
+	  {
+	    if (src[i] >= 0x80)
+	      {
+		require_decoding = 1;
+		break;
+	      }
+	  }
+      }
 
     if (require_decoding)
       {
 	int bufsize;
 	unsigned char *buf;
 	struct coding_system coding;
+	Lisp_Object coding_system = Qnil;
+	
+	/* `next-selection-coding-system' should override everything,
+	   even when the locale passed by the system disagrees.  The
+	   only exception is when `next-selection-coding-system'
+	   requested CF_UNICODETEXT and we couldn't get that. */
+	if (use_configured_coding_system
+	    && !NILP (Vnext_selection_coding_system))
+	    coding_system = Vnext_selection_coding_system;
+
+	/* If we have CF_TEXT or CF_OEMTEXT, we want to check out
+	   CF_LOCALE, too. */
+	else if (actual_clipboard_type != CF_UNICODETEXT)
+	  {
+	    HGLOBAL hlocale;
+	    LCID lcid = DEFAULT_LCID;
+	    UINT cp;
+
+	    /* Documentation says that the OS always generates
+	       CF_LOCALE info automatically, so the locale handle
+	       should always be present.  Fact is that this is not
+	       always true on 9x ;-(.  */
+	    hlocale = GetClipboardData (CF_LOCALE);
+	    if (hlocale != NULL)
+	      {
+		const LCID * lcid_ptr;
+		lcid_ptr = (const LCID *) GlobalLock (hlocale);
+		if (lcid_ptr != NULL)
+		  {
+		    lcid = *lcid_ptr;
+		    GlobalUnlock (hlocale);
+		  }
+
+		/* 9x has garbage as the sort order (to be exact there
+		   is another instance of the language id in the upper
+		   word).  We don't care about sort order anyway, so
+		   we just filter out the unneeded mis-information to
+		   avoid irritations. */
+		lcid = MAKELCID (LANGIDFROMLCID (lcid), SORT_DEFAULT);
+	      }
+
+	    /* If we are using fallback from CF_UNICODETEXT, we can't
+	       use the configured coding system.  Also we don't want
+	       to use it, if the system has supplied us with a locale
+	       and it is not just the system default. */
+	    if (!use_configured_coding_system || lcid != DEFAULT_LCID)
+	      {
+		cp = cp_from_locale (lcid, actual_clipboard_type);
+		/* If it's just our current standard setting anyway,
+		   use the coding system that the user has selected.
+		   Otherwise create a new spec to match the locale
+		   that was specified by the other side or the
+		   system.  */
+		if (!use_configured_coding_system || cp != cfg_codepage)
+		  coding_system = coding_from_cp (cp);
+	      }
+	  }
+
+	if (NILP (coding_system))
+	  coding_system = Vselection_coding_system;
+	Vnext_selection_coding_system = Qnil;
 
-	if (NILP (Vnext_selection_coding_system))
-	  Vnext_selection_coding_system = Vselection_coding_system;
-	setup_coding_system
-	  (Fcheck_coding_system (Vnext_selection_coding_system), &coding);
+	setup_coding_system (Fcheck_coding_system (coding_system), &coding);
 	coding.src_multibyte = 0;
 	coding.dst_multibyte = 1;
-	Vnext_selection_coding_system = Qnil;
 	coding.mode |= CODING_MODE_LAST_BLOCK;
 	/* We explicitely disable composition handling because
 	   selection data should not contain any composition
 	   sequence.  */
 	coding.composing = COMPOSITION_DISABLED;
+	/* Force DOS line-ends. */
+	coding.eol_type = CODING_EOL_CRLF;
+
 	bufsize = decoding_buffer_size (&coding, nbytes);
 	buf = (unsigned char *) xmalloc (bufsize);
 	decode_coding (&coding, src, buf, nbytes, bufsize);
@@ -382,10 +935,13 @@ DEFUN ("w32-get-clipboard-data", Fw32_ge
       }
     else
       {
-	/* Need to know final size after CR chars are removed because we
-	   can't change the string size manually, and doing an extra
-	   copy is silly.  Note that we only remove CR when it appears
-	   as part of CRLF.  */
+	/* FIXME: We may want to repeat the code in this branch for
+	   the Unicode case. */
+
+	/* Need to know final size after CR chars are removed because
+	   we can't change the string size manually, and doing an
+	   extra copy is silly.  We only remove CR when it appears as
+	   part of CRLF.  */
 
 	truelen = nbytes;
 	dst = src;
@@ -462,9 +1018,14 @@ and t is the same as `SECONDARY'.  */)
 
       if (OpenClipboard (NULL))
 	{
-	  int format = 0;
-	  while (format = EnumClipboardFormats (format))
-	    if (format == CF_TEXT)
+	  UINT format = 0;
+	  setup_config ();
+	  while ((format = EnumClipboardFormats (format)))
+	    /* Check CF_TEXT in addition to cfg_clipboard_type,
+	       because we can fall back on that if CF_UNICODETEXT is
+	       not available.  Actually a check for CF_TEXT only
+	       should be enough.  */
+	    if (format == cfg_clipboard_type || format == CF_TEXT)
 	      {
 		val = Qt;
 		break;
@@ -476,24 +1037,25 @@ and t is the same as `SECONDARY'.  */)
   return Qnil;
 }
 
+/* One-time init.  Called in the un-dumped Emacs, but not in the
+   dumped version. */
+
 void
 syms_of_w32select ()
 {
-#if 0
-  defsubr (&Sw32_open_clipboard);
-  defsubr (&Sw32_empty_clipboard);
-  defsubr (&Sw32_close_clipboard);
-#endif
   defsubr (&Sw32_set_clipboard_data);
   defsubr (&Sw32_get_clipboard_data);
   defsubr (&Sx_selection_exists_p);
 
   DEFVAR_LISP ("selection-coding-system", &Vselection_coding_system,
 	       doc: /* Coding system for communicating with other programs.
-When sending or receiving text via cut_buffer, selection, and clipboard,
-the text is encoded or decoded by this coding system.
-The default value is `iso-latin-1-dos'.  */);
-  Vselection_coding_system = intern ("iso-latin-1-dos");
+When sending or receiving text via cut_buffer, selection, and
+clipboard, the text is encoded or decoded by this coding system.
+The default value is the current system default encoding on 9x/Me and
+`utf-16le-dos' (Unicode) on NT/W2K/XP. */);  
+  /* The actual value is set dynamically in the dumped Emacs, see
+     below. */
+  Vselection_coding_system = Qnil;
 
   DEFVAR_LISP ("next-selection-coding-system", &Vnext_selection_coding_system,
 	       doc: /* Coding system for the next communication with other programs.
@@ -504,6 +1066,41 @@ set to nil.  */);
   Vnext_selection_coding_system = Qnil;
 
   QCLIPBOARD = intern ("CLIPBOARD");	staticpro (&QCLIPBOARD);
+
+  cfg_coding_system = Qnil;     staticpro (&cfg_coding_system);
+  current_text = Qnil;		staticpro (&current_text);
+  current_coding_system = Qnil; staticpro (&current_coding_system);
+
+  QUNICODE = intern ("utf-16le-dos"); staticpro (&QUNICODE);
+  QANSICP = Qnil; staticpro (&QANSICP);
+  QOEMCP = Qnil;  staticpro (&QOEMCP);
+}
+
+/* One-time init.  Called in the dumped Emacs, but not in the
+   un-dumped version. */
+
+void
+globals_of_w32select ()
+{
+  DEFAULT_LCID = GetUserDefaultLCID ();
+  /* Drop the sort order from the LCID, so we can compare this with
+     CF_LOCALE objects that have the same fix on 9x.  */
+  DEFAULT_LCID = MAKELCID (LANGIDFROMLCID (DEFAULT_LCID), SORT_DEFAULT);
+
+  ANSICP = GetACP ();
+  OEMCP = GetOEMCP ();
+
+  QANSICP = coding_from_cp (ANSICP);
+  QOEMCP = coding_from_cp (OEMCP);
+
+  if (os_subtype == OS_NT)
+    Vselection_coding_system = QUNICODE;
+  else if (inhibit_window_system)
+    Vselection_coding_system = QOEMCP;
+  else
+    Vselection_coding_system = QANSICP;
+
+  clipboard_owner = create_owner ();
 }
 
 /* arch-tag: c96e9724-5eb1-4dad-be07-289f092fd2af
Index: src/s/ms-w32.h
===================================================================
RCS file: /cvsroot/emacs/emacs/src/s/ms-w32.h,v
retrieving revision 1.30
diff -u -p -r1.30 ms-w32.h
--- src/s/ms-w32.h	1 Sep 2003 15:45:58 -0000	1.30
+++ src/s/ms-w32.h	8 Nov 2004 17:25:32 -0000
@@ -477,8 +477,10 @@ extern char *get_emacs_configuration_opt
    must include config.h to pick up this pragma.  */
 
 /* Names must be < 8 bytes */
+#ifdef _MSC_VER
 #pragma data_seg("EMDATA")
 #pragma bss_seg("EMBSS")
+#endif
 
 /* #define FULL_DEBUG */
 /* #define EMACSDEBUG */

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Unicode support for the MS Windows clipboard [new patch]

[Tak Ota]

Benny,

While using emacs for some time (days) the clipboard operation becomes
unavailable and the following error messages appear in the message
buffer.

Copying from emacs and pasting in another application:
  "Error in delayed clipboard rendering: Text is read-only"

Yanking from clipboard into emacs:
  "w32-get-clipboard-data:(text-read-only)"

After restarting the emacs the problem is gone for a while then it
comes back in later time.  Is there a possibility of leaking memory
handles?

-Tak

Mon, 08 Nov 2004 18:24:44 +0100: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> wrote:

> Hi all,
> 
> 
> Now that my papers are in order (;-)), here is a new version of the
> much discussed patch to support the Unicode clipboard on Windows
> (attached).
> 
> Notes:
> 
> - I didn't provide user-level documentation except a short mention in
>   etc/NEWS.  The defaults should work fine and there should be less
>   need to customize this than on X11.
> 
> - There is a short discussion of the use of the customization in
>   comments in the code.  Something similar may be usefull in the ELisp
>   manual, probably in "Window System Selections."
> 
> 
> benny
> 
> 
> 2004-11-08  Benjamin Riefenstahl  <Benjamin.Riefenstahl@epost.de>
> 
> 	* w32select.c: Summary: Thorough rework to implement Unicode
>         clipboard operations and delayed rendering.
> 
>         Drop last_clipboard_text and related code, keep track of
>         ownership via clipboard_owner instead.  Drop old #if0
>         sections.
> 
> 	(DEFAULT_LCID, ANSICP, OEMCP, QUNICODE, QANSICP, QOEMCP)
> 	(clipboard_owner, modifying_clipboard, cfg_coding_system)
> 	(cfg_codepage, cfg_lcid, cfg_clipboard_type, current_text)
> 	(current_coding_system, current_requires_encoding)
> 	(current_num_nls, current_clipboard_type, current_lcid): New
> 	static variables.
> 
> 	(convert_to_handle_as_ascii, convert_to_handle_as_coded)
>         (render, render_all, run_protected, lisp_error_handler)
>         (owner_callback, create_owner, setup_config)
>         (enum_locale_callback, cp_from_locale, coding_from_cp): New
>         local functions.
> 
>         (term_w32select, globals_of_w32select): New global functions.
> 
> 	(Fw32_set_clipboard_data): Ignore parameter FRAME, use
> 	clipboard_owner instead.  Use delayed rendering and provide
> 	all text formats.  Provide CF_LOCALE if necessary.
> 
> 	(Fw32_get_clipboard_data): Handle CF_UNICODETEXT and
> 	CF_LOCALE.  Fall back to CF_TEXT, if CF_UNICODETEXT is not
> 	available.  Force DOS line-ends for decoding.
> 
> 	(Fx_selection_exists_p): Handle CF_UNICODETEXT.
> 
> 	(syms_of_w32select): Init and register new variables.
> 
>         * w32.h: Add prototypes for globals_of_w32select and
>         term_w32select.  Make the neighboring K&R declarations into
>         prototypes, too.
> 
>         * emacs.c: Include w32.h to get function prototypes.
>         (main): Call globals_of_w32select.
> 
>         * w32.c (term_ntproc): Call term_w32select. 
> 
>         * mule-cmds.el (set-locale-environment): Remove call to
>         set-selection-coding-system on Windows.
> 
>         * s/ms-w32.h: Guard MSC-specific #pragmas with an #ifdef.
> 
> 
> 

Re: Unicode support for the MS Windows clipboard [new patch]

[Benjamin Riefenstahl]

Hi Tak,


Tak Ota writes:
> While using emacs for some time (days) the clipboard operation
> becomes unavailable and the following error messages appear in the
> message buffer.
>
> Copying from emacs and pasting in another application:
>   "Error in delayed clipboard rendering: Text is read-only"
>
> Yanking from clipboard into emacs:
>   "w32-get-clipboard-data:(text-read-only)"

Curious error message.  ;-((

> After restarting the emacs the problem is gone for a while then it
> comes back in later time.  Is there a possibility of leaking memory
> handles?

A stress test indicates that the system leaks memory when Emacs puts
data on the clipboard using the new functionality.  I'll try to find
the source of that.


benny

Re: Unicode support for the MS Windows clipboard [new patch]

[Tak Ota]

Is there any reason why this patch cannot make into CVS?  I really
wish it be a part of the next release.

-Tak

Mon, 08 Nov 2004 18:24:44 +0100: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> wrote:

> Hi all,
> 
> 
> Now that my papers are in order (;-)), here is a new version of the
> much discussed patch to support the Unicode clipboard on Windows
> (attached).
> 
> Notes:
> 
> - I didn't provide user-level documentation except a short mention in
>   etc/NEWS.  The defaults should work fine and there should be less
>   need to customize this than on X11.
> 
> - There is a short discussion of the use of the customization in
>   comments in the code.  Something similar may be usefull in the ELisp
>   manual, probably in "Window System Selections."
> 
> 
> benny
> 
> 
> 2004-11-08  Benjamin Riefenstahl  <Benjamin.Riefenstahl@epost.de>
> 
> 	* w32select.c: Summary: Thorough rework to implement Unicode
>         clipboard operations and delayed rendering.
> 
>         Drop last_clipboard_text and related code, keep track of
>         ownership via clipboard_owner instead.  Drop old #if0
>         sections.
> 
> 	(DEFAULT_LCID, ANSICP, OEMCP, QUNICODE, QANSICP, QOEMCP)
> 	(clipboard_owner, modifying_clipboard, cfg_coding_system)
> 	(cfg_codepage, cfg_lcid, cfg_clipboard_type, current_text)
> 	(current_coding_system, current_requires_encoding)
> 	(current_num_nls, current_clipboard_type, current_lcid): New
> 	static variables.
> 
> 	(convert_to_handle_as_ascii, convert_to_handle_as_coded)
>         (render, render_all, run_protected, lisp_error_handler)
>         (owner_callback, create_owner, setup_config)
>         (enum_locale_callback, cp_from_locale, coding_from_cp): New
>         local functions.
> 
>         (term_w32select, globals_of_w32select): New global functions.
> 
> 	(Fw32_set_clipboard_data): Ignore parameter FRAME, use
> 	clipboard_owner instead.  Use delayed rendering and provide
> 	all text formats.  Provide CF_LOCALE if necessary.
> 
> 	(Fw32_get_clipboard_data): Handle CF_UNICODETEXT and
> 	CF_LOCALE.  Fall back to CF_TEXT, if CF_UNICODETEXT is not
> 	available.  Force DOS line-ends for decoding.
> 
> 	(Fx_selection_exists_p): Handle CF_UNICODETEXT.
> 
> 	(syms_of_w32select): Init and register new variables.
> 
>         * w32.h: Add prototypes for globals_of_w32select and
>         term_w32select.  Make the neighboring K&R declarations into
>         prototypes, too.
> 
>         * emacs.c: Include w32.h to get function prototypes.
>         (main): Call globals_of_w32select.
> 
>         * w32.c (term_ntproc): Call term_w32select. 
> 
>         * mule-cmds.el (set-locale-environment): Remove call to
>         set-selection-coding-system on Windows.
> 
>         * s/ms-w32.h: Guard MSC-specific #pragmas with an #ifdef.
> 
> 
> 

Re: Unicode support for the MS Windows clipboard [new patch]

[Jason Rumney]

Tak Ota <Takaaki.Ota@am.sony.com> writes:

> Is there any reason why this patch cannot make into CVS?  I really
> wish it be a part of the next release.

The last mail I have regarding it is about a memory leak you found.
Benjamin was going to look at it, but I haven't heard anything back.


> Mon, 08 Nov 2004 18:24:44 +0100: Benjamin Riefenstahl <Benjamin.Riefenstahl@epost.de> wrote:
>
>> Hi all,
>> 
>> 
>> Now that my papers are in order (;-)), here is a new version of the
>> much discussed patch to support the Unicode clipboard on Windows
>> (attached).
>> 

Re: Unicode support for the MS Windows clipboard [new patch]

[Tak Ota]

I suppose Benjamin is too busy lately.

Tue, 8 Feb 2005 01:04:32 -0800: "Jason Rumney" <jasonr@gnu.org> wrote:

> Tak Ota <Takaaki.Ota@am.sony.com> writes:
> 
> > Is there any reason why this patch cannot make into CVS?  I really
> > wish it be a part of the next release.
> 
> The last mail I have regarding it is about a memory leak you found.
> Benjamin was going to look at it, but I haven't heard anything back.

I suspect there maybe some relationship between clipboard operation
and the memory leak but I don't have conclusive evidence.  Funny thing
is I haven't seen the same clipboard problem recently, I don't
remember when I saw it last time.  I may have changed my pattern of
clipboard usage.  I don't know.

My point is since I've been using Benjamin's patch for a couple of
month and haven't experienced any catastrophic problem I think it is
worth putting his patch into CVS so that more eyes can hunt for the
potentially hidden bug before the next release.

-Tak

> > Mon, 08 Nov 2004 18:24:44 +0100: Benjamin Riefenstahl
> <Benjamin.Riefenstahl@epost.de> wrote:
> >
> >> Hi all,
> >> 
> >> 
> >> Now that my papers are in order (;-)), here is a new version of the
> >> much discussed patch to support the Unicode clipboard on Windows
> >> (attached).
> >> 

Re: Unicode support for the MS Windows clipboard [new patch]

[Jason Rumney]

[-- Attachment #1.1: Type: text/plain, Size: 1070 bytes --]

Tak Ota wrote:

>I suppose Benjamin is too busy lately.
>
>Tue, 8 Feb 2005 01:04:32 -0800: "Jason Rumney" <jasonr@gnu.org> wrote:
>
>  
>
>>Tak Ota <Takaaki.Ota@am.sony.com> writes:
>>
>>    
>>
>>>Is there any reason why this patch cannot make into CVS?  I really
>>>wish it be a part of the next release.
>>>      
>>>
>>The last mail I have regarding it is about a memory leak you found.
>>Benjamin was going to look at it, but I haven't heard anything back.
>>    
>>
>
>I suspect there maybe some relationship between clipboard operation
>and the memory leak but I don't have conclusive evidence.  Funny thing
>is I haven't seen the same clipboard problem recently, I don't
>remember when I saw it last time.  I may have changed my pattern of
>clipboard usage.  I don't know.
>
>My point is since I've been using Benjamin's patch for a couple of
>month and haven't experienced any catastrophic problem I think it is
>worth putting his patch into CVS so that more eyes can hunt for the
>potentially hidden bug before the next release.
>  
>
OK, I've checked it in.


[-- Attachment #1.2: Type: text/html, Size: 1721 bytes --]

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Unicode support for the MS Windows clipboard [new patch]

[Benjamin Riefenstahl]

Hi Tak,


Tak Ota writes:
> I suppose Benjamin is too busy lately.

Sorry about the delay.  I got distracted by that thing they call
"life".  ;-(

> I suspect there maybe some relationship between clipboard operation
> and the memory leak but I don't have conclusive evidence.

I did a systematic code review and couldn't find any problems.

Than I tried to develop a test.  My machine does freeze (more or less)
during a loop of putting data onto the clipboard and retrieving it
with a simple external tool, but only after some hours.  I could not
see relevant memory consumption when I let the loop just run for 100
or 1000 times.

It's possible that some non-memory resource in the NT kernel is
consumed.  It's also possible that it's just hardware problems with my
machine.

Jason Rumney writes:
> OK, I've checked it in.

Thanks. 


benny