From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: "Daniel B." Newsgroups: gmane.emacs.bugs Subject: bug#3735: shell-mode editing, movement, and command entry broken by prompt/output changes Date: Wed, 01 Jul 2009 15:17:59 -0400 Message-ID: <4A4BB667.8090302@smart.net> Reply-To: "Daniel B." , 3735@emacsbugs.donarmstrong.com NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1246507294 19795 80.91.229.12 (2 Jul 2009 04:01:34 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 2 Jul 2009 04:01:34 +0000 (UTC) To: bug-gnu-emacs@gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Thu Jul 02 06:01:26 2009 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1MMDUA-0004EJ-LZ for geb-bug-gnu-emacs@m.gmane.org; Thu, 02 Jul 2009 06:01:26 +0200 Original-Received: from localhost ([127.0.0.1]:53769 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1MMDUA-0000s2-20 for geb-bug-gnu-emacs@m.gmane.org; Thu, 02 Jul 2009 00:01:22 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1MM5vx-0008NA-Tv for bug-gnu-emacs@gnu.org; Wed, 01 Jul 2009 15:57:33 -0400 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1MM5vs-00087M-5D for bug-gnu-emacs@gnu.org; Wed, 01 Jul 2009 15:57:33 -0400 Original-Received: from [199.232.76.173] (port=43062 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1MM5vr-000872-T2 for bug-gnu-emacs@gnu.org; Wed, 01 Jul 2009 15:57:27 -0400 Original-Received: from rzlab.ucr.edu ([138.23.92.77]:50991) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1MM5vr-0007ot-5G for bug-gnu-emacs@gnu.org; Wed, 01 Jul 2009 15:57:27 -0400 Original-Received: from rzlab.ucr.edu (rzlab.ucr.edu [127.0.0.1]) by rzlab.ucr.edu (8.14.3/8.14.3/Debian-5) with ESMTP id n61JvOPI028544; Wed, 1 Jul 2009 12:57:25 -0700 Original-Received: (from debbugs@localhost) by rzlab.ucr.edu (8.14.3/8.14.3/Submit) id n61Jt53C027887; Wed, 1 Jul 2009 12:55:05 -0700 X-Loop: owner@emacsbugs.donarmstrong.com Resent-From: "Daniel B." Resent-To: bug-submit-list@donarmstrong.com Resent-CC: Emacs Bugs Resent-Date: Wed, 01 Jul 2009 19:55:05 +0000 Resent-Message-ID: Resent-Sender: owner@emacsbugs.donarmstrong.com X-Emacs-PR-Message: report 3735 X-Emacs-PR-Package: emacs X-Emacs-PR-Keywords: Original-Received: via spool by submit@emacsbugs.donarmstrong.com id=B.124647776626953 (code B ref -1); Wed, 01 Jul 2009 19:55:05 +0000 Original-Received: (at submit) by emacsbugs.donarmstrong.com; 1 Jul 2009 19:49:26 +0000 X-Spam-Bayes: score:0.5 Bayes not run. spammytokens:Tokens not available. hammytokens:Tokens not available. Original-Received: from lists.gnu.org (lists.gnu.org [199.232.76.165]) by rzlab.ucr.edu (8.14.3/8.14.3/Debian-5) with ESMTP id n61JnE7J026945 for ; Wed, 1 Jul 2009 12:49:16 -0700 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1MM5nt-0008PD-GZ for bug-gnu-emacs@gnu.org; Wed, 01 Jul 2009 15:49:13 -0400 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1MM5nn-0008Hj-PB for bug-gnu-emacs@gnu.org; Wed, 01 Jul 2009 15:49:12 -0400 Original-Received: from [199.232.76.173] (port=42899 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1MM5nn-0008HX-IC for bug-gnu-emacs@gnu.org; Wed, 01 Jul 2009 15:49:07 -0400 Original-Received: from mail.sovenix.net ([216.127.152.227]:3219 helo=sovenix.net) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1MM5nn-0006Tr-2O for bug-gnu-emacs@gnu.org; Wed, 01 Jul 2009 15:49:07 -0400 Original-Received: from 216.1.16.126 [216.1.16.126] by sovenix.net with ESMTP (SMTPD-11.0) id b9e40009104c37e1; Wed, 1 Jul 2009 15:31:32 -0400 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.19) Gecko/20081204 SeaMonkey/1.1.14 X-detected-operating-system: by monty-python.gnu.org: Windows 2000 SP2+, XP SP1+ (seldom 98) X-Greylist: delayed 1864 seconds by postgrey-1.27 at monty-python; Wed, 01 Jul 2009 15:49:05 EDT X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6 (newer, 2) Resent-Date: Wed, 01 Jul 2009 15:57:33 -0400 X-Mailman-Approved-At: Thu, 02 Jul 2009 00:01:18 -0400 X-BeenThere: bug-gnu-emacs@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:29133 Archived-At: Introduction: In shell mode, editing of command output and previous command lines into new command lines and submission of those new command lines are quite broken. For example, editing a previous command's output lines to turn them into a new command line and then typing RET (executing comint-send-input) to try to submit it to the shell frequently submits only part of the line, and C-a and C-e frequently stop somewhere in the middle of the line. These problems severely reduce the usability of shell mode, which, before the breakage, was great for editing command output into related command lines (and editing it in general). Also, note that the partial submission can be very dangerous. For example, trying to edit an output line "one_thing" (e.g., from an "ls /dir" command) into the command line "rm -rf /dir/one_thing" can easily result in the command line "rm -rf /dir/" instead. The problem appears to be because of changes made to distinguish shell-prompt text from command text (and maybe also changes made to highlight error lines in command output. Apparently, boundaries between runs of text with different attributes cause commands to hit those boundaries instead of their normal boundaries (e.g., line boundaries for C-e, C-a, RET (comint-send-input)). Version information: This problem: - did not exist in Emacs 20.7.2, - first appeared somewhere in Emacs 21.x, - exists in Emacs 22.2.21 (NTEmacs and Debian Etch's emacs22 package, version_22.2+2-5), and - exists in Emacs 22.3 (built from the source in http://ftp.gnu.org/pub/gnu/emacs/emacs-22.3.tar.gz as of 2009-06-30). Details: Most of the problems occur when editing a line that originally was a command output line and that is being edited into a new command line, or when typing RET (to run comint-send-input) on such a line to execute the command line. However, some of the problems also occur when editing or when typing RET on a line that was a previously entered command. The main subproblems are: - Typing RET on a line does not submit the whole intended command to the shell: - It tends to submit only manually inserted text, ignoring original text (from command output or a previous command line). - Text inserted by yanking/pasting sometimes gets treated differently than manually inserted text. (It tends to get ignored.) - Treatment of manually inserted and/or pasted text seems to depend on the source of text is it inserted next to. - Text inserted by command completion sometimes gets ignored or causes other text to be ignored. - Some of Emacs' most basic movement commands do not work normally (how they usually work elsewhere in Emacs) and some do not even work consistently in shell mode (do not work the same at different places in a line), even accounting for presumably intended behavior changes related to distinguishing shell prompts from command text: - C-a sometimes does not move to the beginning of the line (for a command-output line) or of the command line (for a previous-command line), and in some places a second consecutive C-a moves further than the first one. - Mostly similarly, C-e sometimes does not move to the end of the line, and in some places a second C-e moves further. - M-f and M-b sometimes move less than to the next word boundary. - C-n sometimes moves only forward within an line, not to the next line. - C-k does not work normally. - C-k sometimes does not kill to the end of the line. - C-k sometimes kills the line break. - (All of those incorrect movements and killing seem to involve boundaries between manually inserted text vs. original text on the line (command output or a submitted previous command line).) Representative Test Cases: - C1: Inserting text causes previous command output text to be ignored: 1. Generate command output to use as command line and then to edit into later, modified command line: 1.0 At command prompt in shell mode ... 1.1 Insert "echo ls". 1.2 Type RET. 1.3 (Emacs submits whole command line "echo ls".) 1.4 (Shell gives output line "ls".) 2. Use command output as command line: 2.1 Go up to output line "ls" (C-p). 2.2 Type RET. 2.3 Note that Emacs submits whole command line "ls" as expected and intended (and as in Emacs 20). 3. Edit command output into command line: 3.1 Repeat steps 1.x and 2.1, or go back up to original output line from 1.4. 3.2 Go to end of output line (C-e). 3.3 Insert " -la". 3.4 Type RET. 3.5 PROBLEM: Note that Emacs submits just " -la", instead of whole command line "ls -la" (as intended, expected, and in Emacs 20). (If you need a more realistic example, use something like "ls" for the initial command, and then at the beginning of an output line insert something like "ls -l " or "emacs " and then type RET.) - C2: Pasting differs from manual inserting in previous command input: 1. Execute command line to re-execute and then edit into modified command line to be executed: 1.0 At command prompt in shell mode ... 1.1 Insert "echo x". 1.2 Type RET. 2. Re-execute previous command line: 2.1 Go up to previous command line "echo x" (C-p C-p). 2.2 Type RET. 2.3 Note that Emacs submits the whole command line "echo x" as expected and intended. (Emacs copies the command text to the current command-prompt line at the bottom of the buffer, and the shell prints "x".) 3. Insert to edit previous command line into modified version: 3.1 Go up to previous command line "echo x" (C-p C-p). 3.2 Go to end of line (C-e). 3.3 Insert "y". 3.4 Type RET. 3.5 Note that Emacs submits whole command line "echo xy" as expected and intended. 4. Paste to edit previous command line into modified version: 4.1 Copy some text from somewhere. (E.g., move to the "e" in "echo", type C-@ (C-SP) (set-mark-command), type C-f, type M-w (command kill-ring-save).) 4.2 Go to previous command line "echo xy". 4.3 Go to end of line (C-e). 4.4 Yank/paste text (C-y). (E.g., resulting in "echo xye".) 4.5 Type RET. 4.6 PROBLEM: Note that Emacs submits nothing, instead of whole command line (e.g., "echo xye") (as intended, expected, and in Emacs 20). 4.7 Go to now-previous command line (e.g., "echo xye"). 4.8 Go to beginning of line. 4.9 Type RET. 4.10 PROBLEM: Note that Emacs submits just "echo xy", instead of whole command line (e.g., "echo xye") (as intended, expected, and in Emacs 20). (Also, of course, note that Emacs submitted something different for the exact same line as in step 4.5.) - C3: C-a, C-e, M-f, M-b don't move normally in edited command line. 1. Run case C2 to set up. 2. Move to beginning of unedited previous command line: 2.1 Go to the end of the previous command line from step 2.1. 2.2 Type C-a (beginning-of-line). 2.3 Note that Emacs moves left to the beginning of the command line (to the right of the shell prompt text), as apparently intended (in Emacs 21+). 3. Move to beginning of edited previous command line: 3.1 Go to the end of the previous command line from step 4.7. 3.2 Type C-a (beginning-of-line). 3.3 PROBLEM: Note that Emacs moves left only to the beginning of the pasted text, instead of to either the beginning of the the command line (as it does for an unedited command) or the beginning of the line (as it does in Emacs 20). 3.4 Type C-a again. 3.5 PROBLEM: Note that Emacs moves further left (to the beginning of the command line), showing that C-a is not idempotent as it usually is in Emacs. 4. Move forwards in line by word boundaries: 4.1 Go to the beginning of the previous command line (see step 3.1). 4.2 Type M-f (forward-word). 4.3 Note that Emacs moves to the space in "echo xye", as expected. 4.4 Type M-f again. 4.5 PROBLEM: Note that Emacs moves to the "e" in "xye", instead of to the newline character after "xye", as expected, as it does in Emacs 20, as it usually does elsewhere. 4.6 Type M-f a third time. 4.7 Note that Emacs now moves to the newline character after "xye" as expected. 5. Move backwards in line by word boundaries: 5.1 Go to the end of the previous command line (see step 3.1). 5.2 Type M-b (backward-word). 5.3 PROBLEM: Note that Emacs moves to the "e" in "xye" instead of to the "x", as expected, as in Emacs 20, and as it usually does elsewhere. 5.4 Type M-b again. 5.5 Note that emacs moves to the "x", as expected. 5.6 Type M-b a third time. 5.7 Note that emacs moves to the "e" in "echo", as expected. - C4: C-k not at end of line kills newline character too in edited output line 1. Execute command line to generate command output to use as command arguments: 1.0 At command prompt in shell mode ... 1.1 Insert some command that gives several output lines (e.g., "ls"). 1.2 Type RET. 1.3 (Emacs submits whole command line and shell gives output lines.) 2. Edit command output into command with arguments: 2.1 Move to beginning of an output line (e.g., C-p, C-a). 2.2 Insert some command to use edited previous output as argument(s) (e.g., "mv "). 2.3 Type C-k to try to kill to end of line (to copy previous command's output line to paste twice back into line and edit into line command arguments). 2.4 PROBLEM: Emacs kills to end of line PLUS the newline character, instead of killing just to the end of line, as it does here in shell mode in Emacs 20 and as it usually does elsewhere. (For a full example use consider this: 1. Execute "ls *.PNG". 2. Get output line "xyz.PNG". 3. Move up to output line and try to edit into "mv xyz.PNG xyz.png": a. Type C-p C-a. b. Insert "mv ". c. Type C-k d. Type C-y SP C-y. e. Edit "PNG" before point(?)/cursor to "png". (Type DEL DEL DEL, insert "png".) With the correct C-k behavior, you get "mv xyz.PNG xyz.png"; with the broken C-k behavior, you get "mv xyz.PNG" NL " xyx.png". 3. Submit the edited command line. a. Type RET. With the correct C-k behavior, you execute "mv xyz.PNG xyx.png"; with the broken C-k behavior, you execute "mv xyz.PNG".) (Example of danger: Trying to compose something like "mv file1 file2 dir/" could easily result in "mv file1 file2", destroying the previous contents of file2.) - C5: C-k kills less than to end of line in edited output line. 1. Do steps 1.x of case C4. 2. Edit command output into command line. 2.1 Move to beginning of an output line (e.g., C-p, C-a). 2.2 Insert some command to use edited previous output as argument(s) (e.g., "rm -rf "). 3. Try to kill command line to yank to command-prompt line. 3.1 Move to the beginning of the line (e.g., C-a). 3.2 Type C-k. 3.3 PROBLEM: Emacs kills (and copies) only to the end of the inserted text, not killing the rest of the text on the line, instead of killing to the end of line as it does here in shell mode in Emacs 20 and as it usually does elsewhere. (For a full example use consider this: 1. Execute "ls /backups" 2. Get some output line "dir_to_delete" 3. Move up to output line and edit to "rm -rf /backups/dir_to_delete": a. Type C-p C-a. b. insert "rm -rf /backups/". 4. Try to copy line, paste into command-prompt line, and execute: a. Type C-a C-k. c. Move to command-prompt line (e.g., M->). d. Type C-y. e. Type RET. With the correct C-k behavior, you execute "rm -rf /backups/dir_to_delete"; with the broken C-k b behavior, you execute "rm -rf /backups/"!) - C6: Command completion prevents resubmission of previous command line: 1. Execute "ls /e". (Get error from shell.) 2. Go to previous command line. 3. Type TAB to complete. 4. Get "ls /etc". 5. Type RET to execute. 6. Emacs submits "" instead of while "ls /etc". 7. Go to beginning of line. 8. Type RET to execute. 6. Emacs submits "ls /e" instead of while "ls /etc". - C7: Variation on C1 and C3: 1. Execute "echo a ; echo b". 2. Type C-p C-p C-a; insert "echo ". 3. Type C-e C-d; insert "x". 4. Type C-e; type RET. 5. Emacs submits nothing instead of "echo axb". 6. Do steps 1 - 4. 7. Type C-a; type RET. 8. Emacs submits "x" instead of "echo axb". 9. Do steps 1 - 4. 7. Type C-a C-b C-a; type RET. 8. Emacs submits "echo " instead of "echo axb". - C8: C-n doesn't go to next line in edited command output: Note: I'm having trouble reproducing this one. The sequence is _close_ to this: 1. Execute "echo A ; echo B ; echo C". 2. Edit to "echo A B C": - Type C-p C-p C-p C-a. - Insert "echo ". - Replace NLs with space with query-replace. (Type M-x, "query-replace", RET, C-q C-j, RET, SP, RET, y, y, n.) 3. Try to move from edited line to next line. - Move up to beginning of edited line (C-p from where query-replace ends, then C-a). - Try to move down to next line with C-n. - When Emacs exhibits the buggy behavior (that is, when whatever unidentified steps are executed trigger it), Emacs moves only to the space in "echo ", instead of moving down to the next line. (Additionally, C-e goes only partway to the end of the line Additional Use Case and Danger Examples: - Example use case broken by broken RET behavior: 1. Execute "ls". 2. Get some output lines listing files (e.g., "a.txt" and "b.txt"). 3. Edit output into a command operating on the files (e.g., C-p C-p C-a, insert "wc " to insert command; C-e C-d SP to gather multiple filenames into the command line). 4. Type RET to try to execute the line ("wc a.txt b.txt"). With the correct RET behavior, you execute "wc a.txt b.txt"; with the broken RET behavior, you execute nothing. Example of danger: Trying to edit output lines "source1" and "source2" to compose and execute something like "tar -cf x.tar source1 source2" then executing "rm -r source1 source2" could easily result in destruction of the not-copied data in source1 and source2. - Example use case broken by broken C-a behavior: 1. Execute a command and get output. 2. Move to an output line and insert a command at the beginning of the line. 3. Join following lines to the current line with C-e C-d. 4. Decide to modify the command further and try to return to the beginning of the line by typing C-a. With the correct C-a behavior, you end up at the beginning of the line; with the broken C-a behavior, you end up somewhere else. Example of danger: Starting to compose something like "rm a b" from output lines including "a" and "b" and then deciding to edit the command into "echo rm a b" could easily result in executing "rm a b". (Note: The following says 22.3.3 apparently because I built 22.3 three times.) In GNU Emacs 22.3.3 (i686-pc-linux-gnu) of 2009-07-01 on willy configured using `configure '--without-x'' Important settings: value of $LC_ALL: nil value of $LC_COLLATE: nil value of $LC_CTYPE: nil value of $LC_MESSAGES: nil value of $LC_MONETARY: nil value of $LC_NUMERIC: nil value of $LC_TIME: nil value of $LANG: en_US.UTF-8 locale-coding-system: utf-8 default-enable-multibyte-characters: t Major mode: Shell Minor modes in effect: shell-dirtrack-mode: t encoded-kbd-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t unify-8859-on-encoding-mode: t utf-translate-cjk-mode: t auto-compression-mode: t line-number-mode: t Recent input: ESC x s h e l l RET e c h o SPC ; SPC DEL DEL a SPC ; SPC e c h o SPC b RET C-p C-a C-p e c h o SPC C-e x C-d RET ESC x r e p o e TAB DEL r t TAB b DEL RET C-z C-x C-f C-g C-x 2 C-x b RET C-x C-f ESC b ESC b ESC b ESC b C-k RET f C-n C-n C-n C-@ C-e ESC w C-x o ESC x r e p e o TAB C-p C-p C-p C-p C-n C-n C-n C-n e c h o SPC a SPC ; SPC e c h o SPC b RET C-p C-p C-a e c h o SPC C-e C-d x RET r e p o TAB C-_ C-_ ESC x r e p TAB e DEL o r TAB RET Recent messages: find-file-read-args: Command attempted to use minibuffer while in minibuffer Quit Loading dired...done Mark set Quit Completing command name... Partially completed Undo! [2 times] Making completion list... Loading help-mode...done