* Please review: rewrite sql-interactive-remove-continuation-prompt
@ 2022-04-29 21:51 Filipp Gunbin
2022-04-29 22:19 ` Filipp Gunbin
2022-04-30 11:53 ` Lars Ingebrigtsen
0 siblings, 2 replies; 7+ messages in thread
From: Filipp Gunbin @ 2022-04-29 21:51 UTC (permalink / raw)
To: emacs-devel
Hi,
After several attempts to make multiline queries work for Apache
Cassandra, I gave up and almost rewrote
sql-interactive-remove-continuation-prompt. It's now more streamline
and self-contained.
I was coding the logic described in docstring below. If I didn't miss
something, then I'd like to install this and see what breaks. If I did,
then what did I miss?
I tested this on multiline queries with Postgres and Cassandra, I don't
have access to other DBs right now (most importantly, Oracle and MySQL /
Maria). I tested sql-send-region (C-c C-r from sql-mode),
sql-accumulate-and-indent (C-j then RET from sql-interactive-mode) and
just typing into the interpreter.
Previously we were anchoring the prompt search regexps to start/end of
string (` and '), but now just search for regexps as they are. Current
regexps in sql.el all anchor to the beginning of line (^, not `). The
change is indended, as prompts may appear anywhere in the output.
Thanks.
Filipp
diff --git a/lisp/progmodes/sql.el b/lisp/progmodes/sql.el
index 5e5f5e13fe..979b743a65 100644
--- a/lisp/progmodes/sql.el
+++ b/lisp/progmodes/sql.el
@@ -3648,94 +3648,69 @@ sql-input-sender
(defvar sql-preoutput-hold nil)
-(defun sql-starts-with-prompt-re ()
- "Anchor the prompt expression at the beginning of the output line.
-Remove the start of line regexp."
- (concat "\\`" comint-prompt-regexp))
-
-(defun sql-ends-with-prompt-re ()
- "Anchor the prompt expression at the end of the output line.
-Match a SQL prompt or a password prompt."
- (concat "\\(?:\\(?:" sql-prompt-regexp "\\)\\|"
- "\\(?:" comint-password-prompt-regexp "\\)\\)\\'"))
-
(defun sql-interactive-remove-continuation-prompt (oline)
"Strip out continuation prompts out of the OLINE.
Added to the `comint-preoutput-filter-functions' hook in a SQL
-interactive buffer. If `sql-output-newline-count' is greater than
-zero, then an output line matching the continuation prompt is filtered
-out. If the count is zero, then a newline is inserted into the output
-to force the output from the query to appear on a new line.
-
-The complication to this filter is that the continuation prompts
-may arrive in multiple chunks. If they do, then the function
-saves any unfiltered output in a buffer and prepends that buffer
-to the next chunk to properly match the broken-up prompt.
-
-If the filter gets confused, it should reset and stop filtering
-to avoid deleting non-prompt output."
-
- ;; continue gathering lines of text iff
- ;; + we know what a prompt looks like, and
- ;; + there is held text, or
- ;; + there are continuation prompt yet to come, or
- ;; + not just a prompt string
+interactive buffer. The complication to this filter is that the
+continuation prompts may arrive in multiple chunks. If they do,
+then the function saves any unfiltered output in a buffer and
+prepends that buffer to the next chunk to properly match the
+broken-up prompt.
+
+The filter goes into play only if something is already
+accumulated, or we're waiting for continuation
+prompts (`sql-output-newline-count' is positive). In this case:
+- Accumulate process output into `sql-preoutput-hold'.
+- Remove any complete prompts / continuation prompts that we're waiting
+ for.
+- In case we're expecting more prompts - return all currently
+ accumulated _complete_ lines, leaving the rest for the next
+ invocation. They will appear in the output immediately. This way we
+ don't accumulate large chunks of data for no reason.
+- If we found all expected prompts - just return all current accumulated
+ data."
(when (and comint-prompt-regexp
- (or (> (length (or sql-preoutput-hold "")) 0)
- (> (or sql-output-newline-count 0) 0)
- (not (or (string-match sql-prompt-regexp oline)
- (and sql-prompt-cont-regexp
- (string-match sql-prompt-cont-regexp oline))))))
-
+ ;; We either already have something held, or expect
+ ;; prompts
+ (or sql-preoutput-hold
+ (and sql-output-newline-count
+ (> sql-output-newline-count 0))))
(save-match-data
- (let (prompt-found last-nl)
-
- ;; Add this text to what's left from the last pass
- (setq oline (concat sql-preoutput-hold oline)
- sql-preoutput-hold "")
-
- ;; If we are looking for multiple prompts
- (when (and (integerp sql-output-newline-count)
- (>= sql-output-newline-count 1))
- ;; Loop thru each starting prompt and remove it
- (let ((start-re (sql-starts-with-prompt-re)))
- (while (and (not (string= oline ""))
- (> sql-output-newline-count 0)
- (string-match start-re oline))
- (setq oline (replace-match "" nil nil oline)
- sql-output-newline-count (1- sql-output-newline-count)
- prompt-found t)))
-
- ;; If we've found all the expected prompts, stop looking
- (if (= sql-output-newline-count 0)
- (setq sql-output-newline-count nil)
-
- ;; Still more possible prompts, leave them for the next pass
- (setq sql-preoutput-hold oline
- oline "")))
-
- ;; If no prompts were found, stop looking
- (unless prompt-found
- (setq sql-output-newline-count nil
- oline (concat oline sql-preoutput-hold)
- sql-preoutput-hold ""))
-
- ;; Break up output by physical lines if we haven't hit the final prompt
- (let ((end-re (sql-ends-with-prompt-re)))
- (unless (and (not (string= oline ""))
- (string-match end-re oline)
- (>= (match-end 0) (length oline)))
- ;; Find everything upto the last nl
- (setq last-nl 0)
- (while (string-match "\n" oline last-nl)
- (setq last-nl (match-end 0)))
- ;; Hold after the last nl, return upto last nl
- (setq sql-preoutput-hold (concat (substring oline last-nl)
- sql-preoutput-hold)
- oline (substring oline 0 last-nl)))))))
+ ;; Add this text to what's left from the last pass
+ (setq oline (concat sql-preoutput-hold oline)
+ sql-preoutput-hold nil)
+
+ ;; If we are looking for prompts
+ (when (and sql-output-newline-count
+ (> sql-output-newline-count 0))
+ ;; Loop thru each starting prompt and remove it
+ (while (and (not (string-empty-p oline))
+ (> sql-output-newline-count 0)
+ (string-match comint-prompt-regexp oline))
+ (setq oline (replace-match "" nil nil oline)
+ sql-output-newline-count (1- sql-output-newline-count)))
+
+ ;; If we've found all the expected prompts, stop looking
+ (if (= sql-output-newline-count 0)
+ (setq sql-output-newline-count nil)
+ ;; Still more possible prompts, leave them for the next pass
+ (setq sql-preoutput-hold oline
+ oline "")))
+
+ ;; Lines that are now complete may be passed further
+ (when sql-preoutput-hold
+ (let ((last-nl 0))
+ (while (string-match "\n" sql-preoutput-hold last-nl)
+ (setq last-nl (match-end 0)))
+ ;; Return up to last nl, hold after the last nl
+ (setq oline (substring sql-preoutput-hold 0 last-nl)
+ sql-preoutput-hold (substring sql-preoutput-hold last-nl))
+ (when (string-empty-p sql-preoutput-hold)
+ (setq sql-preoutput-hold nil))))))
oline)
+
;;; Sending the region to the SQLi buffer.
(defvar sql-debug-send nil
"Display text sent to SQL process pragmatically.")
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: Please review: rewrite sql-interactive-remove-continuation-prompt
2022-04-29 21:51 Please review: rewrite sql-interactive-remove-continuation-prompt Filipp Gunbin
@ 2022-04-29 22:19 ` Filipp Gunbin
2022-04-30 11:53 ` Lars Ingebrigtsen
1 sibling, 0 replies; 7+ messages in thread
From: Filipp Gunbin @ 2022-04-29 22:19 UTC (permalink / raw)
To: emacs-devel
On 30/04/2022 00:51 +0300, Filipp Gunbin wrote:
> I tested this on multiline queries with Postgres and Cassandra,
Just to note: I used the yet unreleased version (latest commit) of my
sql-cassandra package from ELPA, updated for this patch.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Please review: rewrite sql-interactive-remove-continuation-prompt
2022-04-29 21:51 Please review: rewrite sql-interactive-remove-continuation-prompt Filipp Gunbin
2022-04-29 22:19 ` Filipp Gunbin
@ 2022-04-30 11:53 ` Lars Ingebrigtsen
2022-05-02 16:29 ` Filipp Gunbin
1 sibling, 1 reply; 7+ messages in thread
From: Lars Ingebrigtsen @ 2022-04-30 11:53 UTC (permalink / raw)
To: emacs-devel
Filipp Gunbin <fgunbin@fastmail.fm> writes:
> Previously we were anchoring the prompt search regexps to start/end of
> string (` and '), but now just search for regexps as they are. Current
> regexps in sql.el all anchor to the beginning of line (^, not `). The
> change is indended, as prompts may appear anywhere in the output.
I think that makes change, but before pushing, can you add some test
cases for these things?
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Please review: rewrite sql-interactive-remove-continuation-prompt
2022-04-30 11:53 ` Lars Ingebrigtsen
@ 2022-05-02 16:29 ` Filipp Gunbin
2022-05-03 10:35 ` Lars Ingebrigtsen
0 siblings, 1 reply; 7+ messages in thread
From: Filipp Gunbin @ 2022-05-02 16:29 UTC (permalink / raw)
To: Lars Ingebrigtsen; +Cc: michael, emacs-devel
On 30/04/2022 13:53 +0200, Lars Ingebrigtsen wrote:
> Filipp Gunbin <fgunbin@fastmail.fm> writes:
>
>> Previously we were anchoring the prompt search regexps to start/end of
>> string (` and '), but now just search for regexps as they are. Current
>> regexps in sql.el all anchor to the beginning of line (^, not `). The
>> change is indended, as prompts may appear anywhere in the output.
>
> I think that makes change, but before pushing, can you add some test
> cases for these things?
Sure, added tests.
I've also CC'ed Michael, maybe he has something to say.
Thanks.
commit dfb2707195aea71470f615649057dbd4f9671639
Author: Filipp Gunbin <fgunbin@fastmail.fm>
Date: Sat Apr 30 00:49:04 2022 +0300
Rewrite sql-interactive-remove-continuation-prompt
* lisp/progmodes/sql.el (sql-starts-with-prompt-re): Remove.
(sql-ends-with-prompt-re): Remove
(sql-interactive-remove-continuation-prompt): Delete prompts from
anywhere in the process output, not just at the beginning of current
string. Streamline logic, describe it in docstring.
* test/lisp/progmodes/sql-tests.el: Add tests
diff --git a/lisp/progmodes/sql.el b/lisp/progmodes/sql.el
index 5e5f5e13fe..979b743a65 100644
--- a/lisp/progmodes/sql.el
+++ b/lisp/progmodes/sql.el
@@ -3648,94 +3648,69 @@ sql-input-sender
(defvar sql-preoutput-hold nil)
-(defun sql-starts-with-prompt-re ()
- "Anchor the prompt expression at the beginning of the output line.
-Remove the start of line regexp."
- (concat "\\`" comint-prompt-regexp))
-
-(defun sql-ends-with-prompt-re ()
- "Anchor the prompt expression at the end of the output line.
-Match a SQL prompt or a password prompt."
- (concat "\\(?:\\(?:" sql-prompt-regexp "\\)\\|"
- "\\(?:" comint-password-prompt-regexp "\\)\\)\\'"))
-
(defun sql-interactive-remove-continuation-prompt (oline)
"Strip out continuation prompts out of the OLINE.
Added to the `comint-preoutput-filter-functions' hook in a SQL
-interactive buffer. If `sql-output-newline-count' is greater than
-zero, then an output line matching the continuation prompt is filtered
-out. If the count is zero, then a newline is inserted into the output
-to force the output from the query to appear on a new line.
-
-The complication to this filter is that the continuation prompts
-may arrive in multiple chunks. If they do, then the function
-saves any unfiltered output in a buffer and prepends that buffer
-to the next chunk to properly match the broken-up prompt.
-
-If the filter gets confused, it should reset and stop filtering
-to avoid deleting non-prompt output."
-
- ;; continue gathering lines of text iff
- ;; + we know what a prompt looks like, and
- ;; + there is held text, or
- ;; + there are continuation prompt yet to come, or
- ;; + not just a prompt string
+interactive buffer. The complication to this filter is that the
+continuation prompts may arrive in multiple chunks. If they do,
+then the function saves any unfiltered output in a buffer and
+prepends that buffer to the next chunk to properly match the
+broken-up prompt.
+
+The filter goes into play only if something is already
+accumulated, or we're waiting for continuation
+prompts (`sql-output-newline-count' is positive). In this case:
+- Accumulate process output into `sql-preoutput-hold'.
+- Remove any complete prompts / continuation prompts that we're waiting
+ for.
+- In case we're expecting more prompts - return all currently
+ accumulated _complete_ lines, leaving the rest for the next
+ invocation. They will appear in the output immediately. This way we
+ don't accumulate large chunks of data for no reason.
+- If we found all expected prompts - just return all current accumulated
+ data."
(when (and comint-prompt-regexp
- (or (> (length (or sql-preoutput-hold "")) 0)
- (> (or sql-output-newline-count 0) 0)
- (not (or (string-match sql-prompt-regexp oline)
- (and sql-prompt-cont-regexp
- (string-match sql-prompt-cont-regexp oline))))))
-
+ ;; We either already have something held, or expect
+ ;; prompts
+ (or sql-preoutput-hold
+ (and sql-output-newline-count
+ (> sql-output-newline-count 0))))
(save-match-data
- (let (prompt-found last-nl)
-
- ;; Add this text to what's left from the last pass
- (setq oline (concat sql-preoutput-hold oline)
- sql-preoutput-hold "")
-
- ;; If we are looking for multiple prompts
- (when (and (integerp sql-output-newline-count)
- (>= sql-output-newline-count 1))
- ;; Loop thru each starting prompt and remove it
- (let ((start-re (sql-starts-with-prompt-re)))
- (while (and (not (string= oline ""))
- (> sql-output-newline-count 0)
- (string-match start-re oline))
- (setq oline (replace-match "" nil nil oline)
- sql-output-newline-count (1- sql-output-newline-count)
- prompt-found t)))
-
- ;; If we've found all the expected prompts, stop looking
- (if (= sql-output-newline-count 0)
- (setq sql-output-newline-count nil)
-
- ;; Still more possible prompts, leave them for the next pass
- (setq sql-preoutput-hold oline
- oline "")))
-
- ;; If no prompts were found, stop looking
- (unless prompt-found
- (setq sql-output-newline-count nil
- oline (concat oline sql-preoutput-hold)
- sql-preoutput-hold ""))
-
- ;; Break up output by physical lines if we haven't hit the final prompt
- (let ((end-re (sql-ends-with-prompt-re)))
- (unless (and (not (string= oline ""))
- (string-match end-re oline)
- (>= (match-end 0) (length oline)))
- ;; Find everything upto the last nl
- (setq last-nl 0)
- (while (string-match "\n" oline last-nl)
- (setq last-nl (match-end 0)))
- ;; Hold after the last nl, return upto last nl
- (setq sql-preoutput-hold (concat (substring oline last-nl)
- sql-preoutput-hold)
- oline (substring oline 0 last-nl)))))))
+ ;; Add this text to what's left from the last pass
+ (setq oline (concat sql-preoutput-hold oline)
+ sql-preoutput-hold nil)
+
+ ;; If we are looking for prompts
+ (when (and sql-output-newline-count
+ (> sql-output-newline-count 0))
+ ;; Loop thru each starting prompt and remove it
+ (while (and (not (string-empty-p oline))
+ (> sql-output-newline-count 0)
+ (string-match comint-prompt-regexp oline))
+ (setq oline (replace-match "" nil nil oline)
+ sql-output-newline-count (1- sql-output-newline-count)))
+
+ ;; If we've found all the expected prompts, stop looking
+ (if (= sql-output-newline-count 0)
+ (setq sql-output-newline-count nil)
+ ;; Still more possible prompts, leave them for the next pass
+ (setq sql-preoutput-hold oline
+ oline "")))
+
+ ;; Lines that are now complete may be passed further
+ (when sql-preoutput-hold
+ (let ((last-nl 0))
+ (while (string-match "\n" sql-preoutput-hold last-nl)
+ (setq last-nl (match-end 0)))
+ ;; Return up to last nl, hold after the last nl
+ (setq oline (substring sql-preoutput-hold 0 last-nl)
+ sql-preoutput-hold (substring sql-preoutput-hold last-nl))
+ (when (string-empty-p sql-preoutput-hold)
+ (setq sql-preoutput-hold nil))))))
oline)
+
;;; Sending the region to the SQLi buffer.
(defvar sql-debug-send nil
"Display text sent to SQL process pragmatically.")
diff --git a/test/lisp/progmodes/sql-tests.el b/test/lisp/progmodes/sql-tests.el
index 7e36d845e2..c644d115df 100644
--- a/test/lisp/progmodes/sql-tests.el
+++ b/test/lisp/progmodes/sql-tests.el
@@ -425,5 +425,85 @@ sql-tests-comint-automatic-password
(let ((sql-password "password"))
(should (equal "password" (sql-comint-automatic-password "")))))
+
+\f
+;; Tests for sql-interactive-remove-continuation-prompt
+
+(defmacro sql-tests-remove-cont-prompts-harness (&rest body)
+ "Set-up and tear-down for tests of
+`sql-interactive-remove-continuation-prompt'."
+ (declare (indent 0))
+ `(let ((comint-prompt-regexp "^ +\\.\\{3\\} ")
+ (sql-output-newline-count nil)
+ (sql-preoutput-hold nil))
+ ,@body
+ (should (null sql-output-newline-count))
+ (should (null sql-preoutput-hold))))
+
+(ert-deftest sql-tests-remove-cont-prompts-pass-through ()
+ "Test that `sql-interactive-remove-continuation-prompt' just
+passes the output line through when it doesn't expect prompts."
+ (sql-tests-remove-cont-prompts-harness
+ (should
+ (equal " ... "
+ (sql-interactive-remove-continuation-prompt
+ " ... ")))))
+
+(ert-deftest sql-tests-remove-cont-prompts-anchored-successive ()
+ "Test that `sql-interactive-remove-continuation-prompt' is able
+to delete multiple prompts (anchored to bol) even if they appear
+in a single line, but not more than `sql-output-newline-count'."
+ (sql-tests-remove-cont-prompts-harness
+ (setq sql-output-newline-count 2)
+ (should
+ (equal
+ ;; 2 of 3 prompts are deleted
+ "some output ... more output...\n\
+ ... \n\
+output after prompt"
+ (sql-interactive-remove-continuation-prompt
+ "some output ... more output...\n\
+ ... ... ... \n\
+output after prompt")))))
+
+(ert-deftest sql-tests-remove-cont-prompts-collect-chunked-output ()
+ "Test that `sql-interactive-remove-continuation-prompt' properly
+collects output when output arrives in chunks, with prompts
+intermixed."
+ (sql-tests-remove-cont-prompts-harness
+ (setq sql-output-newline-count 2)
+
+ ;; Part of first prompt gets held. Complete line is passed
+ ;; through.
+ (should (equal "line1\n"
+ (sql-interactive-remove-continuation-prompt
+ "line1\n ..")))
+ (should (equal " .." sql-preoutput-hold))
+ (should (equal 2 sql-output-newline-count))
+
+ ;; First prompt is complete - remove it. Hold part of line2.
+ (should (equal ""
+ (sql-interactive-remove-continuation-prompt ". li")))
+ (should (equal "li" sql-preoutput-hold))
+ (should (equal 1 sql-output-newline-count))
+
+ ;; Remove second prompt. Flush output & don't hold / process any
+ ;; output further on.
+ (should (equal "line2\nli"
+ (sql-interactive-remove-continuation-prompt "ne2\n ... li")))
+ (should (null sql-preoutput-hold))
+ (should (null sql-output-newline-count))
+ (should (equal "line3\n ... "
+ (sql-interactive-remove-continuation-prompt "line3\n ... ")))))
+
+(ert-deftest sql-tests-remove-cont-prompts-flush-held ()
+ "Test that when we don't wait for prompts,
+ `sql-interactive-remove-continuation-prompt' just 'flushes' held
+ output, with no prompt processing."
+ (sql-tests-remove-cont-prompts-harness
+ (setq sql-preoutput-hold "line1\n ..")
+ (should (equal "line1\n ... line2 .."
+ (sql-interactive-remove-continuation-prompt ". line2 ..")))))
+
(provide 'sql-tests)
;;; sql-tests.el ends here
^ permalink raw reply related [flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-05-03 11:19 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-04-29 21:51 Please review: rewrite sql-interactive-remove-continuation-prompt Filipp Gunbin
2022-04-29 22:19 ` Filipp Gunbin
2022-04-30 11:53 ` Lars Ingebrigtsen
2022-05-02 16:29 ` Filipp Gunbin
2022-05-03 10:35 ` Lars Ingebrigtsen
2022-05-03 11:16 ` Filipp Gunbin
2022-05-03 11:19 ` Lars Ingebrigtsen
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).