From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Po Lu Newsgroups: gmane.emacs.devel Subject: Re: textconv.c Date: Sun, 12 Feb 2023 22:05:27 +0800 Message-ID: <87k00nyo60.fsf@yahoo.com> References: <83r0uvghw7.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="3340"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Feb 12 15:08:58 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pRD2I-0000jZ-Gj for ged-emacs-devel@m.gmane-mx.org; Sun, 12 Feb 2023 15:08:58 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pRD1V-000395-9J; Sun, 12 Feb 2023 09:08:09 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pRD1P-00038b-Li for emacs-devel@gnu.org; Sun, 12 Feb 2023 09:08:04 -0500 Original-Received: from sonic313-35.consmr.mail.ne1.yahoo.com ([66.163.185.58]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pRD1N-0002s6-80 for emacs-devel@gnu.org; Sun, 12 Feb 2023 09:08:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1676210876; bh=n85FU6kGFe4iy3oV/AgIkowucKp5bRm+bxCbS94VYPE=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From:Subject:Reply-To; b=E6+TGZsXBDyFC7y/eB3GzoPWBjNMp8hP2ha0HrgJqQT5eyCA7hqZoal9OHxtjWE6dHjd4toonGPzTjoySTa2IS3jirNJxS/EjXCIMs7zfw6NZzj2RzlkqFH4c5j89D4j/u50UHqKOWq6br84oP4ymqQnQRz+03rl7CQGk0JCBUZjdsvw+yn5ZEOxk8kcYJ/EQm+yUena0CUPxG1pIOONSEDkiavF7Qjzo/Z8SKNqA/W/PvcgEGdnBGrTXL6I6kqBPj38SeIQxsiK0R0btnTTWHCvA38BLXLEMjDxUQDkIOp1bJxTpTB/p1gavW2fthue+i5d8/i7VNPXC8SzZiJBNw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1676210876; bh=atEif6R5HBP3YRjzNmF0iMnJhN6Fkt3A6DUSNF7JVTo=; h=X-Sonic-MF:From:To:Subject:Date:From:Subject; b=HsiT0nLARgjSxvNCxDv5UQMOADFBk3EfMTXVlfHAR4U6VHsPbo0g24352AYyM2pPkBYIss3w64md444WwxKOOAl+RLZKUbAQjdLkUxMzaZUvpqMybHSXk+40o56FgbNTYPx6hrXlpii3GcR6e+XgaJsBjVAbmXK25EDTM+zD2Vse/6u69fmJkVg3aRJKcSGvAb6TTuQvopH+J6xof4ZHs6bkqFbc0CdICCnn+74+oIK9o+0k6f5SGv1n2dv4siebnjZf37WHMO2qGPM/zxOch3m9kdEgQjxfwAh82INFwxE3l4y0OMBzRnbBUozjQPdcXKaM1t/ZCSBjoSAwkJXW5Q== X-YMail-OSG: THOoD4kVM1mOVwQCFP3i0c79gUt_ypgUoBeiqhmI7o_xdQolJzePT3p2Mrldbby xdWRl32oE6ZXaemfWzPb.Oq0xP12Q4tcQq0zPVxV61yDQiOnKt1NpmS7DJ8Q30YDCiQT4hAYhkm7 00PyUIa3JdU4xkYd8tC0cTdP7MgMOHj9jLppZjNftoGphP.YKZQKEdzs17VrbgfltA8K66yv_njc mZz2LaYD2hIvg8aChTYKxtWn6wVczx3ESV0y.mFE3KTA929MVPJoygjjoQPqiDrtgT0r5AB8cYvz 6oZuKZzGYvSToIQ7hxFqmBwKaKU3aaDEVy0YdMWh6H8ZjYeNqDgYjE101BE6FONJ5e_G8Au13wva y_L3pOCqcERGQvwgtD.ncMn7pNo.Z2uYPnIeFR5WvwhUNfKec8OImgS05aIG1js3shAP_m.3YVNd Te5oJoJxXtX3kNjt6lIUeFcitn7VDOzWlzt8Pqj7m_NM8oTU6QlNyjtkqOCjLyfA1rbYs.zx_qmD X9Jkk27z6MwXYO4SWWbXFwJYPtU_9_ng7DJOdxujrAOLHHTDQgoVxxyBpdzHhlOs1XCgsonoGIP0 o1LRlTQuETq1XJBzP8.wn49CFeH4uLPObgaJUBqUXHS.yxJ_c3PSwcd0d.UAS6a7zjSQULGPu1tJ 9zW1hwAy0bbJlIsracWscw3xTkMzMaNlv3gt_tqGoUP.7e8rQ6PjvQsnTC5dPSnyEMFv7zd1H1Sf JQWiR4YqI0ISy2viExDrSl_8RDsGqU0qopAS.O48TN3OS.3j8HFOuYI30W8l8Kr4xOwYOViHzUCb KFB_IGczSUioyYBLzrMI9BhuHl2xPEKbnbOJjYzgRG X-Sonic-MF: Original-Received: from sonic.gate.mail.ne1.yahoo.com by sonic313.consmr.mail.ne1.yahoo.com with HTTP; Sun, 12 Feb 2023 14:07:56 +0000 Original-Received: by hermes--production-sg3-9fc5746c8-vmkgs (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 848a7fb503659e002b5fd34c22754d34; Sun, 12 Feb 2023 14:05:52 +0000 (UTC) In-Reply-To: <83r0uvghw7.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 12 Feb 2023 14:58:16 +0200") X-Mailer: WebService/1.1.21183 mail.backend.jedi.jws.acl:role.jedi.acl.token.atz.jws.hermes.yahoo Received-SPF: pass client-ip=66.163.185.58; envelope-from=luangruo@yahoo.com; helo=sonic313-35.consmr.mail.ne1.yahoo.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:303176 Archived-At: Eli Zaretskii writes: > Please move the code which manipulates buffer text and the gap to > insdel.c. And I don't really understand why you needed to add > copy_buffer, since we already have insert_from_buffer and copy_text. > > The rest (textconv_query) should go to some existing file(fns.c, > perhaps?). There's no justification for a new file for such a little > code. > > Thanks. That code is going to be significantly expanded in the feature/android branch, so I expect it will become much larger (and definitely worth its own file.) `insert_from_buffer' inserts text into a buffer, instead of copying out of a buffer, and `copy_text' seems to not handle the gap nor work on character positions. The work on the feature/android branch will later be useful for PGTK as well. I implemented this feature on X in order to have an implementation of text conversion on a free operating system. > P.S. I wonder why you couldn't ask about this before doing this stuff. > This list is supposed to be used for such design discussions. I didn't think the small amount of code that was installed amounted to a significant design change. There will be many more changes on the feature/android branch, since Android input methods more or less can't work without the ability to perform arbitrary modifications to buffer text, and the ability to obtain buffer contents and the position of the point in the buffer being displayed. My idea is that on each redisplay after the point in a frame's selected window changes, and each change to buffer contents in the selected window around point, 600 characters (which is the absolute minimum required to satisfy the input method-side buffer content analysis) around point will be transferred to the thread which runs the Android GUI and be made available to input methods, and any editing requests from input methods will be sent back to Emacs, translated into ``text conversion'' requests, and then performed in the selected window's buffer. textconv.c will contain the code that figures out what text to report to the input method, and how to carry out its editing requests (i.e. whether or not to convert text insertion and deletion into key events, or to insert that text directly into the buffer.) I have a sinking suspicion that this is going to be non trivial. I tried five or six variations of using a standalone buffer for character compositions from Android input methods before concluding that such an approach can not work, because input methods demand text they insert to later appear inside a query for the contents of the buffer, and to be able to later delete the text themselves. Basically, the problem is this: if you type h e l l o SPC and the input method inserts ``hello '', it will remember that it inserted ``hello '' at the current position of point. When you later press backspace, instead of sending a delete key event, the input method just asks you to delete the text that was where it thinks ``hello '' is. In effect, this means that when you type DEL, Emacs actually receives: ``replace text from 0 to 6 with the preconversion text hello'' and if you keep pressing DEL after that, Emacs doesn't get any delete key events, but rather: ``replace the preconversion text with hell'' ``replace the preconversion text with hel'' ``replace the preconversion text he'' ``replace the preconversion text h'' ``delete the preconversion text'' until the word ``hello'' disappears from the buffer. Then, instead of sending more delete key events, the input method asks for 600 characters before the current position of the cursor, analyzes the text there, and performs the same process on each character, until it cannot find any more text, at which point it stops sending any events altogether. Likewise, when inserting text, the input method does not send any key events. Instead, when you type ``hello'' and finally SPC, the input method simply tells Emacs to: ``tell me where the point is'' suppose Emacs responds with the number ``42''. The input method will then ask: ``give me 100 before 42, and then 5 characters after 42'' at which point it performs analysis on both segments of extracted text in order to determine whether or not to punctuate the word, or even to insert any text at all. Finally, it asks: ``insert the text hello after 42'' which Emacs is supposed to obey. The same applies in a lesser degree to newer input methods on Wayland and X11 systems. The current support we have there is quite buggy and does not work with input methods designed for handhelds, and input methods which perform any kind of analysis on text surrounding a buffer did not work at all before the change I installed. I stayed up very late last night trying several different approaches to the problem before concluding that for input methods to work on Android (and probably Wayland based systems after several years), Emacs must provide a 100% by-the-book implementation of what input methods expect. Here are two examples of what approaches I tried, and how it blew up. If you provide an editing buffer that is always completely empty, the input method refuses to send any events at all. The input method which comes with the Android emulator complains loudly about Emacs ``misbehaving'' and not reporting back changes made by the input method itself. The third party input method AnySoftKeyboard doesn't do that, but insteads assumes Emacs is buggy and proceeds to operate on the text it thinks it inserted, so when you hit DEL it ends up sending the word you previously inserted with a single letter chopped off the end, instead of a DEL key event. If you provide an editing buffer that contains any preconversion text, but clears itself and sends its contents to Emacs every time preconversion ends, then AnySoftKeyboard simply ignores the buffer being cleared, while the Android emulator's input method locks up for a while to re-initialize itself. The emulator's input method always inserts spaces after the word itself, and without signalling any kind of preconversion, and it does not insert spaces after it locks up, in effect making it impossible to insert any spaces at all. I tried more approaches, but I can't remember what exactly they were or why they didn't work.