bug#18240: 24.3; sql-postgres password with cygwin psql

bug#18240: 24.3; sql-postgres password with cygwin psql

[Mark Wilkinson]

[-- Attachment #1: Type: text/plain, Size: 3504 bytes --]

Starting emacs 24.3.1 with -Q, sql.el version: 3.4

1. Set sql-postgres-program to psql from cygwin:

     (setq sql-postgres-program "c:/cygwin/bin/psql.exe")

2. Run sql-postgres
3. At prompts, enter user (postgres), database (postgres), server
   (localhost).

4. Wait for sql-delay seconds. Login...done.
   Buffer is *SQL* (SQLi[Postgres]:run).
   Prompt for password still not seen.  This is the problem.
   psql prompt not seen.

5. <RET> in that buffer yields

     Password for user postgres:
     psql: fe_sendauth: no password supplied

   and buffer's process dies.

However, repeating steps 2 through 4, doing send-invisible and
entering the password, the process remains and accepts input and
produces psql output (still no psql prompt, though). This does not
work with Windows psql.exe.

This also occurs in emacs-24-20140526-154828-w32-bin where sql.el 3.4
is the default version.



In GNU Emacs 24.3.1 (i386-mingw-nt5.1.2600)
 of 2013-03-17 on MARVIN
Windowing system distributor `Microsoft Corp.', version 5.1.2600
Configured using:
 `configure --with-gcc (4.7) --cflags
 -ID:/devel/emacs/libs/libXpm-3.5.8/include
 -ID:/devel/emacs/libs/libXpm-3.5.8/src
 -ID:/devel/emacs/libs/libpng-dev_1.4.3-1/include
 -ID:/devel/emacs/libs/zlib-dev_1.2.5-2/include
 -ID:/devel/emacs/libs/giflib-4.1.4-1/include
 -ID:/devel/emacs/libs/jpeg-6b-4/include
 -ID:/devel/emacs/libs/tiff-3.8.2-1/include
 -ID:/devel/emacs/libs/gnutls-3.0.9/include
 -ID:/devel/emacs/libs/libiconv-1.13.1-1-dev/include
 -ID:/devel/emacs/libs/libxml2-2.7.8/include/libxml2'

Important settings:
  value of $EMACSDATA: C:/emacs-24.3/etc
  value of $EMACSDOC: C:/emacs-24.3/etc
  value of $EMACSLOADPATH:
C:/emacs-24.3/site-lisp;C:/emacs-24.3/../site-lisp;C:/emacs-24.3/lisp;C:/emacs-24.3/leim
  value of $EMACSPATH: C:/emacs-24.3/bin
  value of $LANG: ENU
  locale-coding-system: cp1252
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<escape> x r e p o r t SPC e m a c s SPC b SPC <re
turn>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
c:/emacs-24.3/site-lisp/sql hides c:/emacs-24.3/lisp/progmodes/sql

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils time-date tooltip ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 ls-lisp w32-common-fns disp-table w32-win
w32-vars tool-bar dnd fontset image regexp-opt fringe tabulated-list
newcomment lisp-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese hebrew greek romanian slovak czech european ethiopic
indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple
abbrev minibuffer loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
w32 multi-tty emacs)

[-- Attachment #2: Type: text/html, Size: 3956 bytes --]

bug#18240: sql-postgres does not prompt for password

[Matthew Smiglarski]

This bug does not happen on the 24.3 release but does happen on 24.3.93.

The problem is with sql-interactive-remove-continuation-prompt, one of
the functions configured to be called by comint-output-filter.

It is caused by code introduced by this git revision.

commit 34499a8eb3c10db8c79a7aa87e1bbce0ae499fb6
Author: Michael Mauger <mmaug@yahoo.com>
Date:   Tue Jul 23 20:25:53 2013 -0400

One workaround is to type the password in anyway. Another workaround
is to remove some of the code with the following patch:

diff --git a/lisp/progmodes/sql.el b/lisp/progmodes/sql.el
index 13d4178..2fd755d 100644
--- a/lisp/progmodes/sql.el
+++ b/lisp/progmodes/sql.el
@@ -3337,18 +3337,7 @@ to avoid deleting non-prompt output."
         (unless prompt-found
           (setq sql-output-newline-count nil
                 oline (concat oline sql-preoutput-hold)
-                sql-preoutput-hold ""))
-
-        ;; Break up output by physical lines if we haven't hit the final prompt
-        (unless (and (not (string= oline ""))
-                     (string-match (sql-ends-with-prompt-re) oline)
-                     (>= (match-end 0) (length oline)))
-          (setq last-nl 0)
-          (while (string-match "\n" oline last-nl)
-            (setq last-nl (match-end 0)))
-          (setq sql-preoutput-hold (concat (substring oline last-nl)
-                                           sql-preoutput-hold)
-                oline (substring oline 0 last-nl))))))
+                sql-preoutput-hold "")))))
    oline)

This sql-interactive-remove-continuation-prompt function removes
continuation prompt (the prompt sent when halfway through a command)
and this improves the usability of multi-line SQL statements on the
interpreter.

There is another small bug with that code where the continuation line
sneaks through, throwing the alignment out.

By this, I mean the "template1->" prompt in the following examples:

template1=> select 42;
 ?column?
----------
       42
(1 row)

template1=> select
42;
template1->  ?column?
----------
       42
(1 row)

template1=> select
42,
43;
template1->  ?column? | ?column?
----------+----------
       42 |       43
(1 row)

That happens because of the following lines:

        ;; Add this text to what's left from the last pass
        (setq oline (concat sql-preoutput-hold oline)
              sql-preoutput-hold "")

I hope this helps.

bug#18240: 24.3; sql-postgres password with cygwin psql

[Filipp Gunbin]

It's not clear what this code is supposed to do...  However it breaks
entering passwords.  Maybe just remove it?

        ;; Break up output by physical lines if we haven't hit the final prompt
        (unless (and (not (string= oline ""))
                     (string-match (sql-ends-with-prompt-re) oline)
                     (>= (match-end 0) (length oline)))
          (setq last-nl 0)
          (while (string-match "\n" oline last-nl)
            (setq last-nl (match-end 0)))
          (setq sql-preoutput-hold (concat (substring oline last-nl)
                                           sql-preoutput-hold)
                oline (substring oline 0 last-nl))))))

-- 
    Filipp

bug#18240: sql-postgres does not prompt for password

[Michael Mauger]

[-- Attachment #1.1: Type: text/plain, Size: 3091 bytes --]

I apologize for the delay,I've been off-line for a bit.  I was aware of the failure in some edge cases and I think I have a solution.  I've attached an elisp file that you should be able to require once sql.el has been loaded that overrides the function as it appears in sql.el.
Please give it a try and see if it corrects the behavior you see.
Thanks
Michael
 

     On Monday, September 22, 2014 4:10 AM, Matthew Smiglarski <matthew.smiglarski@gmail.com> wrote:
   
 

 This bug does not happen on the 24.3 release but does happen on 24.3.93.

The problem is with sql-interactive-remove-continuation-prompt, one of
the functions configured to be called by comint-output-filter.

It is caused by code introduced by this git revision.

commit 34499a8eb3c10db8c79a7aa87e1bbce0ae499fb6
Author: Michael Mauger <mmaug@yahoo.com>
Date:  Tue Jul 23 20:25:53 2013 -0400

One workaround is to type the password in anyway. Another workaround
is to remove some of the code with the following patch:

diff --git a/lisp/progmodes/sql.el b/lisp/progmodes/sql.el
index 13d4178..2fd755d 100644
--- a/lisp/progmodes/sql.el
+++ b/lisp/progmodes/sql.el
@@ -3337,18 +3337,7 @@ to avoid deleting non-prompt output."
        (unless prompt-found
          (setq sql-output-newline-count nil
                oline (concat oline sql-preoutput-hold)
-                sql-preoutput-hold ""))
-
-        ;; Break up output by physical lines if we haven't hit the final prompt
-        (unless (and (not (string= oline ""))
-                    (string-match (sql-ends-with-prompt-re) oline)
-                    (>= (match-end 0) (length oline)))
-          (setq last-nl 0)
-          (while (string-match "\n" oline last-nl)
-            (setq last-nl (match-end 0)))
-          (setq sql-preoutput-hold (concat (substring oline last-nl)
-                                          sql-preoutput-hold)
-                oline (substring oline 0 last-nl))))))
+                sql-preoutput-hold "")))))
    oline)

This sql-interactive-remove-continuation-prompt function removes
continuation prompt (the prompt sent when halfway through a command)
and this improves the usability of multi-line SQL statements on the
interpreter.

There is another small bug with that code where the continuation line
sneaks through, throwing the alignment out.

By this, I mean the "template1->" prompt in the following examples:

template1=> select 42;
 ?column?
----------
      42
(1 row)

template1=> select
42;
template1->  ?column?
----------
      42
(1 row)

template1=> select
42,
43;
template1->  ?column? | ?column?
----------+----------
      42 |      43
(1 row)

That happens because of the following lines:

        ;; Add this text to what's left from the last pass
        (setq oline (concat sql-preoutput-hold oline)
              sql-preoutput-hold "")

I hope this helps.


 
   

[-- Attachment #1.2: Type: text/html, Size: 5401 bytes --]

[-- Attachment #2: sql-interactive-remove-continuation-prompt.el --]
[-- Type: application/octet-stream, Size: 5143 bytes --]

;;; sql-interactive-remove-continuation-prompt.el --- Replacement for sql-interactive-remove-continuation-prompt in sql.el  -*- lexical-binding: t; -*-

;; Copyright (C) 2014  Michael R. Mauger

;; Author: Michael R. Mauger <michael@mauger.com>
;; Keywords: processes, languages

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program.  If not, see <http://www.gnu.org/licenses/>.

;;; Commentary:

;; Test replacement for sql-interactive-remove-continuation-prompt
;; found in sql.el 3.4 found in Emacs 24.4.

;;; Code:

;;; Strip out continuation prompts

(defvar sql-preoutput-hold nil)

(defun sql-starts-with-prompt-re ()
  "Anchor the prompt expression at the beginning of the output line.
Remove the start of line regexp."
  (replace-regexp-in-string "\\^" "\\\\`" comint-prompt-regexp))

(defun sql-ends-with-prompt-re ()
  "Anchor the prompt expression at the end of the output line.
Remove the start of line regexp from the prompt expression since
it may not follow newline characters in the output line."
  (concat (replace-regexp-in-string "\\^" "" sql-prompt-regexp) "\\'"))

(defun sql-interactive-remove-continuation-prompt (oline)
  "Strip out continuation prompts out of the OLINE.

Added to the `comint-preoutput-filter-functions' hook in a SQL
interactive buffer.  If `sql-output-newline-count' is greater than
zero, then an output line matching the continuation prompt is filtered
out.  If the count is zero, then a newline is inserted into the output
to force the output from the query to appear on a new line.

The complication to this filter is that the continuation prompts
may arrive in multiple chunks.  If they do, then the function
saves any unfiltered output in a buffer and prepends that buffer
to the next chunk to properly match the broken-up prompt.

If the filter gets confused, it should reset and stop filtering
to avoid deleting non-prompt output."

  ;; continue gathering lines of text iff
  ;;  + we know what a prompt looks like, and
  ;;  + there is held text, or
  ;;  + there are continuation prompt yet to come, or
  ;;  + not just a prompt string
  (when (and comint-prompt-regexp
             (or (> (length (or sql-preoutput-hold "")) 0)
                 (> (or sql-output-newline-count 0) 0)
                 (not (or (string-match sql-prompt-regexp oline)
                          (string-match sql-prompt-cont-regexp oline)))))

    (save-match-data
      (let (prompt-found last-nl)

        ;; Add this text to what's left from the last pass
        (setq oline (concat sql-preoutput-hold oline)
              sql-preoutput-hold "")

        ;; If we are looking for multiple prompts
        (when (and (integerp sql-output-newline-count)
                   (>= sql-output-newline-count 1))
          ;; Loop thru each starting prompt and remove it
          (let ((start-re (sql-starts-with-prompt-re)))
            (while (and (not (string= oline ""))
                      (> sql-output-newline-count 0)
                      (string-match start-re oline))
              (setq oline (replace-match "" nil nil oline)
                    sql-output-newline-count (1- sql-output-newline-count)
                    prompt-found t)))

          ;; If we've found all the expected prompts, stop looking
          (if (= sql-output-newline-count 0)
              (setq sql-output-newline-count nil
                    oline (concat "\n" oline))

            ;; Still more possible prompts, leave them for the next pass
            (setq sql-preoutput-hold oline
                  oline "")))

        ;; If no prompts were found, stop looking
        (unless prompt-found
          (setq sql-output-newline-count nil
                oline (concat oline sql-preoutput-hold)
                sql-preoutput-hold ""))

        ;; Break up output by physical lines if we haven't hit the final prompt
        (let ((end-re (sql-ends-with-prompt-re)))
          (unless (and (not (string= oline ""))
                       (string-match end-re oline)
                       (>= (match-end 0) (length oline)))
            ;; Find everything upto the last nl
            (setq last-nl 0)
            (while (string-match "\n" oline last-nl)
              (setq last-nl (match-end 0)))
            ;; Hold after the last nl, return upto last nl
            (setq sql-preoutput-hold (concat (substring oline last-nl)
                                             sql-preoutput-hold)
                  oline (substring oline 0 last-nl)))))))
  oline)

(provide 'sql-interactive-remove-continuation-prompt)
;;; sql-interactive-remove-continuation-prompt.el ends here

bug#18240: sql-postgres does not prompt for password

[Matthew Smiglarski]

[-- Attachment #1: Type: text/plain, Size: 4052 bytes --]

Hello, and thanks for the reply.

I tried the patch on trunk and found that the continuation lines are
shown and there is no interference with the alignment of the tabular
output, however the password prompt still isn't presented until the
after the password is entered.

I have attached 2 screenshots:

* emacs24.3.1-sql-postgres-password-prompt.png
Shows the password prompt in the previous version.

* emacs-trunk-with-patch.png
Shows the behaviour I found both before and after your patch.
While the continuation line output is different, you can see that the
password prompt comes after the user input both before and after
eval-ing your patch. (To be clear, I eval your patch about halfway
down, as indicated in the comment beginning "-- Evaluated ...".)

Admittedly this isn't a huge problem to me because I know the
workaround where typing in the password unprompted will log me in
anyway, instead displaying the prompt after I hit return!

Thanks,
Matthew

On Fri, Oct 31, 2014 at 8:37 PM, Michael Mauger <michael@mauger.com> wrote:
> I apologize for the delay,I've been off-line for a bit.  I was aware of the
> failure in some edge cases and I think I have a solution.  I've attached an
> elisp file that you should be able to require once sql.el has been loaded
> that overrides the function as it appears in sql.el.
>
> Please give it a try and see if it corrects the behavior you see.
>
> Thanks
>
> Michael
>
>
> On Monday, September 22, 2014 4:10 AM, Matthew Smiglarski
> <matthew.smiglarski@gmail.com> wrote:
>
>
>
> This bug does not happen on the 24.3 release but does happen on 24.3.93.
>
> The problem is with sql-interactive-remove-continuation-prompt, one of
> the functions configured to be called by comint-output-filter.
>
> It is caused by code introduced by this git revision.
>
> commit 34499a8eb3c10db8c79a7aa87e1bbce0ae499fb6
> Author: Michael Mauger <mmaug@yahoo.com>
> Date:  Tue Jul 23 20:25:53 2013 -0400
>
> One workaround is to type the password in anyway. Another workaround
> is to remove some of the code with the following patch:
>
> diff --git a/lisp/progmodes/sql.el b/lisp/progmodes/sql.el
> index 13d4178..2fd755d 100644
> --- a/lisp/progmodes/sql.el
> +++ b/lisp/progmodes/sql.el
> @@ -3337,18 +3337,7 @@ to avoid deleting non-prompt output."
>         (unless prompt-found
>           (setq sql-output-newline-count nil
>                 oline (concat oline sql-preoutput-hold)
> -                sql-preoutput-hold ""))
> -
> -        ;; Break up output by physical lines if we haven't hit the final
> prompt
> -        (unless (and (not (string= oline ""))
> -                    (string-match (sql-ends-with-prompt-re) oline)
> -                    (>= (match-end 0) (length oline)))
> -          (setq last-nl 0)
> -          (while (string-match "\n" oline last-nl)
> -            (setq last-nl (match-end 0)))
> -          (setq sql-preoutput-hold (concat (substring oline last-nl)
> -                                          sql-preoutput-hold)
> -                oline (substring oline 0 last-nl))))))
> +                sql-preoutput-hold "")))))
>     oline)
>
> This sql-interactive-remove-continuation-prompt function removes
> continuation prompt (the prompt sent when halfway through a command)
> and this improves the usability of multi-line SQL statements on the
> interpreter.
>
> There is another small bug with that code where the continuation line
> sneaks through, throwing the alignment out.
>
> By this, I mean the "template1->" prompt in the following examples:
>
> template1=> select 42;
> ?column?
> ----------
>       42
> (1 row)
>
> template1=> select
> 42;
> template1->  ?column?
> ----------
>       42
> (1 row)
>
> template1=> select
> 42,
> 43;
> template1->  ?column? | ?column?
> ----------+----------
>       42 |      43
> (1 row)
>
> That happens because of the following lines:
>
>         ;; Add this text to what's left from the last pass
>         (setq oline (concat sql-preoutput-hold oline)
>               sql-preoutput-hold "")
>
> I hope this helps.
>
>

[-- Attachment #2: emacs-trunk-with-patch.png --]
[-- Type: image/png, Size: 66667 bytes --]

[-- Attachment #3: emacs24.3.1-sql-postgres-password-prompt.png --]
[-- Type: image/png, Size: 45045 bytes --]