Tramp and conversion of \r\n into \n

Tramp and conversion of \r\n into \n

[yyoncho]

[-- Attachment #1: Type: text/plain, Size: 1974 bytes --]

Hi Michael,

I am writing this regarding the following issue on lsp-mode's bug tracker:
https://github.com/emacs-lsp/lsp-mode/issues/2375 . The issue is that \r\n
are converted to \n but only when connecting to linux from mac.

Using (set-process-coding-system result 'no-conversion 'no-conversion) and
settinginhibit-eol-conversion to t does not work(or at least this is what
was reported by the users).

Can you suggest what the issue might be?

Let me know if you need more information: Here it is the relevant code
which we use for opening the connection:

 (let* ((final-command (lsp-resolve-final-function local-command))
                          ;; wrap with stty to disable converting \r to \n
                          (process-name (generate-new-buffer-name name))
                          (wrapped-command (append '("stty" "raw" ";")
                                                   final-command
                                                   (list
                                                    (concat "2>"
                                                            (or (when
generate-error-file-fn
                                                                  (funcall
generate-error-file-fn name))
                                                                (format
"/tmp/%s-%s-stderr" name

(cl-incf lsp--stderr-index)))))))
                          (process-environment
                           (lsp--compute-process-environment
environment-fn)))
                     (let ((proc (apply 'start-file-process-shell-command
process-name
                                        (format "*%s*" process-name)
wrapped-command)))
                       (set-process-sentinel proc sentinel)
                       (set-process-filter proc filter)
                       (set-process-query-on-exit-flag proc nil)
                       (set-process-coding-system proc 'binary 'binary)
                       (cons proc proc)))

Thanks,
Ivan

[-- Attachment #2: Type: text/html, Size: 3711 bytes --]

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

yyoncho <yyoncho@gmail.com> writes:

> Hi Michael,

Hi Ivan,

> I am writing this regarding the following issue on lsp-mode's bug
> tracker: https://github.com/emacs-lsp/lsp-mode/issues/2375 . The issue
> is that \r\n are converted to \n but only when connecting to linux
> from mac.
>
> Using (set-process-coding-system result 'no-conversion 'no-conversion)
> and settinginhibit-eol-conversion to t does not work(or at least this
> is what was reported by the users).
>
> Can you suggest what the issue might be?

Hard to say (and I have no Mac for testing). Maybe you bring me in
contact with a user beaten by this, and I'll try to see with her, what
happens. That means reproduce the problem while there are Tramp traces,
and likely to repeat the tests several times, applying patches proposed
by me.

> Thanks,
> Ivan

Best regards, Michael.

Re: Tramp and conversion of \r\n into \n

[APEL Martin]

[-- Attachment #1: Type: text/plain, Size: 1299 bytes --]

Hi all,

I am one of the people affected by an issue, which comes from somewhere between lsp-mode and TRAMP. In my case it is between two Linux systems. I opened an issue in the bug tracker for lsp-mode (https://github.com/emacs-lsp/lsp-mode/issues/2375), which describes the issue in more detail.
I am happy to provide additional information, if I can.

Best Regards,

Martin


This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systèmes does not accept or assume any liability or responsibility for any use of or reliance on this email.


Please be informed that your personal data are processed according to our data privacy policy as described on our website. Should you have any questions related to personal data protection, please contact 3DS Data Protection Officer at 3DS.compliance-privacy@3ds.com<mailto:3DS.compliance-privacy@3ds.com>


For other languages, go to https://www.3ds.com/terms/email-disclaimer

[-- Attachment #2: Type: text/html, Size: 4454 bytes --]

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

APEL Martin <Martin.APEL@3ds.com> writes:

> Hi all,

Hi Martin,

> I am one of the people affected by an issue, which comes from
> somewhere between lsp-mode and TRAMP. In my case it is between two
> Linux systems. I opened an issue in the bug tracker for lsp-mode
> (https://github.com/emacs-lsp/lsp-mode/issues/2375), which describes
> the issue in more detail.
>
> I am happy to provide additional information, if I can.

Thanks for the offer. I would like to debug it myself, because I expect
subtle Tramp problems which might not be seen even in the
traces. However, I'm a n00b wrt lsp-mode. Could you pls give me a short
"lsp-mode for dummies", that I can setup and test myself? I've started
yesterday already, but I've failed to find proper lsp server and client
installation instructions.

I'm running Fedora 33, if that matters.

> Best Regards,
>
> Martin

Best regards, Michael.

RE: Tramp and conversion of \r\n into \n

[APEL Martin]

Hi Michael,

I could offer you a test case for a C++ environment:
I have a script to build clangd from the LLVM sources and the necessary Emacs configuration settings. You would obviously need two machines (which could be virtual), where you open a C++ file in Emacs, which should then initialize the clangd on the remote machine. Would that be a case you would want to investigate?

Best Regards,

Martin

-----Original Message-----
From: Michael Albinus <michael.albinus@gmx.de>
Sent: 11 December 2020 10:15
To: APEL Martin <Martin.APEL@3ds.com>
Cc: emacs-devel@gnu.org
Subject: Re: Tramp and conversion of \r\n into \n

EXTERNAL EMAIL : The sender of this email is external to 3DS. Be wary of the content and do not open unexpected attachments or links. If you consider this email as spam, you can click the following link https://spam-report.3ds.com/?linktext=https://www.mailcontrol.com/sr/TJpj9FUOr6DGX2PQPOmvUkBugPZ_3WFC2u7WT0tMn32uoFJW_sajxF-lQqUVbq5wJrqVYUl7c0xzr_jqbv_p6A==  (no login or additional action will be requested).


APEL Martin <Martin.APEL@3ds.com> writes:

> Hi all,

Hi Martin,

> I am one of the people affected by an issue, which comes from
> somewhere between lsp-mode and TRAMP. In my case it is between two
> Linux systems. I opened an issue in the bug tracker for lsp-mode
> (https://github.com/emacs-lsp/lsp-mode/issues/2375), which describes
> the issue in more detail.
>
> I am happy to provide additional information, if I can.

Thanks for the offer. I would like to debug it myself, because I expect subtle Tramp problems which might not be seen even in the traces. However, I'm a n00b wrt lsp-mode. Could you pls give me a short "lsp-mode for dummies", that I can setup and test myself? I've started yesterday already, but I've failed to find proper lsp server and client installation instructions.

I'm running Fedora 33, if that matters.

> Best Regards,
>
> Martin

Best regards, Michael.
This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systèmes does not accept or assume any liability or responsibility for any use of or reliance on this email.


Please be informed that your personal data are processed according to our data privacy policy as described on our website. Should you have any questions related to personal data protection, please contact 3DS Data Protection Officer at 3DS.compliance-privacy@3ds.com<mailto:3DS.compliance-privacy@3ds.com>


For other languages, go to https://www.3ds.com/terms/email-disclaimer

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

APEL Martin <Martin.APEL@3ds.com> writes:

> Hi Michael,

Hi Martin,

> I could offer you a test case for a C++ environment:
> I have a script to build clangd from the LLVM sources and the
> necessary Emacs configuration settings. You would obviously need two
> machines (which could be virtual), where you open a C++ file in Emacs,
> which should then initialize the clangd on the remote machine. Would
> that be a case you would want to investigate?

Yes, pls send. There's also <https://snapcraft.io/install/clangd/fedora>,
would this be sufficient?

I don't lack other machines, running Tramp tests permanently :-)

> Best Regards,
>
> Martin

Best regards, Michael.

RE: Tramp and conversion of \r\n into \n

[APEL Martin]

[-- Attachment #1: Type: text/plain, Size: 3611 bytes --]

Hi Michael,

I created a small test setup as follows:
- A mini C++ project
- An Emacs Lisp file with initializations for LSP mode
- A build script to create clangd
- A wrapper script for clangd, so you can see the output of clangd

Steps to do:
1. Setup clangd: The clangd version you mentioned is rather old, but you can try, if lsp mode works with that version. Otherwise I have added a small build script, which will generate clangd from its sources. It should install clangd somewhere into your path. This needs to be done on the 'remote' machine.
2. Put the wrapper-script somewhere into your path on the 'remote' machine. This simply redirects stdout and stderr of clangd, when it is invoked. This is referenced by the Emacs configuration.
3. Unpack the small example project and run the init.sh script in the project directory on the 'remote' machine. It will run cmake with the necessary options to generate a compile_commands.json file, which is needed by clangd.
4. Install lsp-mode with all its dependencies on the 'local' machine.
5. Make sure the initializations from the .el file are executed.
6. Now try to open the main.cpp file on the remote machine in Emacs on the local machine. This should trigger the start of clangd via lsp-mode. If the error occurs, the modeline shows 'clangd starting' forever.

Hope this helps,

Martin

-----Original Message-----
From: Michael Albinus <michael.albinus@gmx.de>
Sent: 11 December 2020 11:26
To: APEL Martin <Martin.APEL@3ds.com>
Cc: emacs-devel@gnu.org
Subject: Re: Tramp and conversion of \r\n into \n

EXTERNAL EMAIL : The sender of this email is external to 3DS. Be wary of the content and do not open unexpected attachments or links. If you consider this email as spam, you can click the following link https://spam-report.3ds.com/?linktext=https://www.mailcontrol.com/sr/PQhcOnFqQ7HGX2PQPOmvUnWT5wH6X0A_UZF2JwsSf9afotUhni4wvENwzuRffNHjJrqVYUl7c0yhqfUB1i-uFQ==  (no login or additional action will be requested).


APEL Martin <Martin.APEL@3ds.com> writes:

> Hi Michael,

Hi Martin,

> I could offer you a test case for a C++ environment:
> I have a script to build clangd from the LLVM sources and the
> necessary Emacs configuration settings. You would obviously need two
> machines (which could be virtual), where you open a C++ file in Emacs,
> which should then initialize the clangd on the remote machine. Would
> that be a case you would want to investigate?

Yes, pls send. There's also <https://snapcraft.io/install/clangd/fedora>,
would this be sufficient?

I don't lack other machines, running Tramp tests permanently :-)

> Best Regards,
>
> Martin

Best regards, Michael.
This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systèmes does not accept or assume any liability or responsibility for any use of or reliance on this email.


Please be informed that your personal data are processed according to our data privacy policy as described on our website. Should you have any questions related to personal data protection, please contact 3DS Data Protection Officer at 3DS.compliance-privacy@3ds.com<mailto:3DS.compliance-privacy@3ds.com>


For other languages, go to https://www.3ds.com/terms/email-disclaimer

[-- Attachment #2: test_setup.tar.gz --]
[-- Type: application/x-gzip, Size: 1072 bytes --]

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

APEL Martin <Martin.APEL@3ds.com> writes:

> Hi Michael,

Hi Martin,

> I created a small test setup as follows:

Thanks. I will try later today, or tomorrow, as time permits.

> Hope this helps,
>
> Martin

Best regards, Michael.

Re: Tramp and conversion of \r\n into \n

[yyoncho]

[-- Attachment #1: Type: text/plain, Size: 1065 bytes --]

Hi Michael,

You may drop the instructions here and they will follow up.

Thanks,
Ivan

On Thu, Dec 10, 2020 at 3:44 PM Michael Albinus <michael.albinus@gmx.de>
wrote:

> yyoncho <yyoncho@gmail.com> writes:
>
> > Hi Michael,
>
> Hi Ivan,
>
> > I am writing this regarding the following issue on lsp-mode's bug
> > tracker: https://github.com/emacs-lsp/lsp-mode/issues/2375 . The issue
> > is that \r\n are converted to \n but only when connecting to linux
> > from mac.
> >
> > Using (set-process-coding-system result 'no-conversion 'no-conversion)
> > and settinginhibit-eol-conversion to t does not work(or at least this
> > is what was reported by the users).
> >
> > Can you suggest what the issue might be?
>
> Hard to say (and I have no Mac for testing). Maybe you bring me in
> contact with a user beaten by this, and I'll try to see with her, what
> happens. That means reproduce the problem while there are Tramp traces,
> and likely to repeat the tests several times, applying patches proposed
> by me.
>
> > Thanks,
> > Ivan
>
> Best regards, Michael.
>

[-- Attachment #2: Type: text/html, Size: 1703 bytes --]

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Martin,

>> I created a small test setup as follows:
>
> Thanks. I will try later today, or tomorrow, as time permits.

I'm sorry, but it will take some other days. I have implemented a new
Tramp feature, which has damaged my local Tramp such a way it is not
working anymore. So I need to debug this first ...

>> Hope this helps,
>>
>> Martin

Best regards, Michael.

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

APEL Martin <Martin.APEL@3ds.com> writes:

> Hi Michael,

Hi Martin,

I've followd your recipe. My local machine runs Fedora 33, the remote
machine runs Ubuntu 20.10.

> Steps to do:
> 1. Setup clangd: The clangd version you mentioned is rather old, but
> you can try, if lsp mode works with that version. Otherwise I have
> added a small build script, which will generate clangd from its
> sources. It should install clangd somewhere into your path. This needs
> to be done on the 'remote' machine.

I have installed the Ubuntu package of clangd. It is

--8<---------------cut here---------------start------------->8---
# apt list clangd
Listing... Done
clangd/groovy,now 1:11.0-51~exp1 amd64 [installed]
clangd/groovy 1:11.0-51~exp1 i386
--8<---------------cut here---------------end--------------->8---

> 2. Put the wrapper-script somewhere into your path on the 'remote' machine. This simply redirects stdout and stderr of clangd, when it is invoked. This is referenced by the Emacs configuration.

Done. clangd path name adapted to /usr/bin/clangd.

> 3. Unpack the small example project and run the init.sh script in the project directory on the 'remote' machine. It will run cmake with the necessary options to generate a compile_commands.json file, which is needed by clangd.

Done.

> 4. Install lsp-mode with all its dependencies on the 'local' machine.

Done. I've taken lsp-mode 20201209.1825 from MELPA.

> 5. Make sure the initializations from the .el file are executed.

Done.

> 6. Now try to open the main.cpp file on the remote machine in Emacs on the local machine. This should trigger the start of clangd via lsp-mode. If the error occurs, the modeline shows 'clangd starting' forever.

No error happened, everything looks normal. What else could I do?

> Hope this helps,
>
> Martin

Best regards, Michael.

RE: Tramp and conversion of \r\n into \n

[APEL Martin]

Hi Michael,

thanks for your feedback. I am using more or less the same version of lsp-mode. In my case the local machine is a Debian Buster installation, whereas the remote machine runs CentOS 7.8. Have you checked, if the /tmp/clangd-remote file contains any error message? When you move the cursor onto the function name or onto the std::cout, do you see any output, either as a popup or in the minibuffer?

Best Regards,

Martin

> -----Original Message-----
> From: Michael Albinus <michael.albinus@gmx.de>
> Sent: 15 December 2020 12:38
> To: APEL Martin <Martin.APEL@3ds.com>
> Cc: emacs-devel@gnu.org
> Subject: Re: Tramp and conversion of \r\n into \n
>
> APEL Martin <Martin.APEL@3ds.com> writes:
>
> > Hi Michael,
>
> Hi Martin,
>
> I've followd your recipe. My local machine runs Fedora 33, the remote
> machine runs Ubuntu 20.10.
>
> > Steps to do:
> > 1. Setup clangd: The clangd version you mentioned is rather old, but
> > you can try, if lsp mode works with that version. Otherwise I have
> > added a small build script, which will generate clangd from its
> > sources. It should install clangd somewhere into your path. This needs
> > to be done on the 'remote' machine.
>
> I have installed the Ubuntu package of clangd. It is
>
> --8<---------------cut here---------------start------------->8---
> # apt list clangd
> Listing... Done
> clangd/groovy,now 1:11.0-51~exp1 amd64 [installed] clangd/groovy 1:11.0-
> 51~exp1 i386 --8<---------------cut here---------------end--------------->8---
>
> > 2. Put the wrapper-script somewhere into your path on the 'remote'
> machine. This simply redirects stdout and stderr of clangd, when it is invoked.
> This is referenced by the Emacs configuration.
>
> Done. clangd path name adapted to /usr/bin/clangd.
>
> > 3. Unpack the small example project and run the init.sh script in the project
> directory on the 'remote' machine. It will run cmake with the necessary
> options to generate a compile_commands.json file, which is needed by
> clangd.
>
> Done.
>
> > 4. Install lsp-mode with all its dependencies on the 'local' machine.
>
> Done. I've taken lsp-mode 20201209.1825 from MELPA.
>
> > 5. Make sure the initializations from the .el file are executed.
>
> Done.
>
> > 6. Now try to open the main.cpp file on the remote machine in Emacs on
> the local machine. This should trigger the start of clangd via lsp-mode. If the
> error occurs, the modeline shows 'clangd starting' forever.
>
> No error happened, everything looks normal. What else could I do?
>
> > Hope this helps,
> >
> > Martin
>
> Best regards, Michael.
This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systèmes does not accept or assume any liability or responsibility for any use of or reliance on this email.


Please be informed that your personal data are processed according to our data privacy policy as described on our website. Should you have any questions related to personal data protection, please contact 3DS Data Protection Officer at 3DS.compliance-privacy@3ds.com<mailto:3DS.compliance-privacy@3ds.com>


For other languages, go to https://www.3ds.com/terms/email-disclaimer

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

[-- Attachment #1: Type: text/plain, Size: 548 bytes --]

APEL Martin <Martin.APEL@3ds.com> writes:

> Hi Michael,

Hi Martin,

> Have you checked, if the /tmp/clangd-remote file contains any error
> message?

I have appended the (remote) /tmp/clangd-error and /tmp/clangd-logs.

> When you move the cursor onto the function name or onto the
> std::cout, do you see any output, either as a popup or in the
> minibuffer?

Yes, in the minibuffer.

FTR, I'm using Emacs 28.0.50 and Tramp 2.5.0-pre, both from git
master. I don't know whether this matters.

> Best Regards,
>
> Martin

Best regards, Michael.


[-- Attachment #2: clangd-error --]
[-- Type: application/octet-stream, Size: 7265 bytes --]

I[12:55:35.575] Ubuntu clangd version 11.0.0-2
I[12:55:35.576] PID: 212437
I[12:55:35.576] Working directory: /home/albinus/tmp/minicpp
I[12:55:35.576] argv[0]: /usr/bin/clangd
I[12:55:35.576] Starting LSP over stdin/stdout
I[12:55:35.592] <-- initialize(13)
I[12:55:35.593] --> reply:initialize(13) 1 ms
I[12:55:35.601] <-- initialized
I[12:55:35.606] <-- textDocument/didOpen
I[12:55:35.608] Loaded compilation database from /home/albinus/tmp/minicpp
I[12:55:35.608] --> window/workDoneProgress/create(0)
I[12:55:35.609] Enqueueing 1 commands for indexing
I[12:55:35.609] ASTWorker building file /home/albinus/tmp/minicpp/main.cpp version 0 with command 
[/home/albinus/tmp/minicpp/obj]
/usr/bin/c++ --driver-mode=g++ -o CMakeFiles/test.dir/main.cpp.o -c /home/albinus/tmp/minicpp/main.cpp -fsyntax-only -resource-dir=/usr/lib/llvm-11/lib/clang/11.0.0
E[12:55:35.616] Could not build a preamble for file /home/albinus/tmp/minicpp/main.cpp version 0
I[12:55:35.623] <-- textDocument/codeAction(14)
I[12:55:35.627] <-- textDocument/documentLink(15)
I[12:55:35.633] <-- reply(0)
I[12:55:35.633] --> $/progress
I[12:55:35.633] --> $/progress
I[12:55:35.696] --> $/progress
I[12:55:35.696] --> $/progress
I[12:55:36.086] --> textDocument/publishDiagnostics
I[12:55:36.086] --> reply:textDocument/codeAction(14) 463 ms
I[12:55:36.086] --> reply:textDocument/documentLink(15) 459 ms
I[15:30:21.824] <-- textDocument/codeAction(16)
I[15:30:21.825] --> reply:textDocument/codeAction(16) 0 ms
I[15:30:21.828] <-- textDocument/documentLink(17)
I[15:30:21.829] --> reply:textDocument/documentLink(17) 0 ms
I[15:30:24.222] <-- textDocument/codeAction(18)
I[15:30:24.223] --> reply:textDocument/codeAction(18) 0 ms
I[15:30:24.226] <-- textDocument/documentLink(19)
I[15:30:24.226] --> reply:textDocument/documentLink(19) 0 ms
I[15:30:24.233] <-- textDocument/documentHighlight(20)
I[15:30:24.234] --> reply:textDocument/documentHighlight(20) 0 ms
I[15:30:24.239] <-- textDocument/hover(21)
I[15:30:24.263] --> reply:textDocument/hover(21) 23 ms
I[15:30:29.722] <-- textDocument/codeAction(22)
I[15:30:29.722] --> reply:textDocument/codeAction(22) 0 ms
I[15:30:29.726] <-- textDocument/documentLink(23)
I[15:30:29.726] --> reply:textDocument/documentLink(23) 0 ms
I[15:30:29.734] <-- textDocument/documentHighlight(24)
I[15:30:29.734] --> reply:textDocument/documentHighlight(24) 0 ms
I[15:30:29.740] <-- textDocument/hover(25)
I[15:30:29.742] --> reply:textDocument/hover(25) 1 ms
I[15:30:33.373] <-- textDocument/codeAction(26)
I[15:30:33.374] --> reply:textDocument/codeAction(26) 0 ms
I[15:30:33.379] <-- textDocument/documentLink(27)
I[15:30:33.380] --> reply:textDocument/documentLink(27) 0 ms
I[15:30:33.388] <-- textDocument/documentHighlight(28)
I[15:30:33.388] --> reply:textDocument/documentHighlight(28) 0 ms
I[15:30:33.394] <-- textDocument/hover(29)
I[15:30:33.395] --> reply:textDocument/hover(29) 0 ms
I[15:30:36.934] <-- textDocument/codeAction(30)
I[15:30:36.935] <-- textDocument/documentLink(31)
I[15:30:36.935] --> reply:textDocument/codeAction(30) 0 ms
I[15:30:36.935] --> reply:textDocument/documentLink(31) 0 ms
I[15:30:36.936] <-- textDocument/documentHighlight(32)
I[15:30:36.936] --> reply:textDocument/documentHighlight(32) 0 ms
I[15:30:36.941] <-- textDocument/hover(33)
I[15:30:36.941] --> reply:textDocument/hover(33) 0 ms
I[15:30:39.215] <-- textDocument/codeAction(34)
I[15:30:39.215] <-- textDocument/documentLink(35)
I[15:30:39.215] --> reply:textDocument/codeAction(34) 0 ms
I[15:30:39.216] --> reply:textDocument/documentLink(35) 0 ms
I[15:30:39.220] <-- textDocument/documentHighlight(36)
I[15:30:39.221] --> reply:textDocument/documentHighlight(36) 0 ms
I[15:30:39.226] <-- textDocument/hover(37)
I[15:30:39.227] --> reply:textDocument/hover(37) 1 ms
I[15:30:42.206] <-- textDocument/codeAction(38)
I[15:30:42.207] <-- textDocument/documentLink(39)
I[15:30:42.207] <-- textDocument/documentHighlight(40)
I[15:30:42.207] <-- textDocument/hover(41)
I[15:30:42.207] --> reply:textDocument/codeAction(38) 0 ms
I[15:30:42.207] --> reply:textDocument/documentLink(39) 0 ms
I[15:30:42.207] --> reply:textDocument/documentHighlight(40) 0 ms
I[15:30:42.208] --> reply:textDocument/hover(41) 1 ms
I[15:30:47.059] <-- textDocument/codeAction(42)
I[15:30:47.060] --> reply:textDocument/codeAction(42) 0 ms
I[15:30:47.066] <-- textDocument/documentLink(43)
I[15:30:47.066] --> reply:textDocument/documentLink(43) 0 ms
I[15:30:47.073] <-- textDocument/documentHighlight(44)
I[15:30:47.074] --> reply:textDocument/documentHighlight(44) 0 ms
I[15:30:47.081] <-- textDocument/hover(45)
I[15:30:47.084] --> reply:textDocument/hover(45) 3 ms
I[15:30:50.850] <-- textDocument/codeAction(46)
I[15:30:50.851] --> reply:textDocument/codeAction(46) 0 ms
I[15:30:50.858] <-- textDocument/documentLink(47)
I[15:30:50.858] --> reply:textDocument/documentLink(47) 0 ms
I[15:30:50.864] <-- textDocument/documentHighlight(48)
I[15:30:50.864] --> reply:textDocument/documentHighlight(48) 0 ms
I[15:30:50.871] <-- textDocument/hover(49)
I[15:30:50.871] --> reply:textDocument/hover(49) 0 ms
I[15:30:54.580] <-- textDocument/codeAction(50)
I[15:30:54.580] --> reply:textDocument/codeAction(50) 0 ms
I[15:30:54.585] <-- textDocument/documentLink(51)
I[15:30:54.585] --> reply:textDocument/documentLink(51) 0 ms
I[15:30:54.591] <-- textDocument/documentHighlight(52)
I[15:30:54.591] --> reply:textDocument/documentHighlight(52) 0 ms
I[15:30:54.597] <-- textDocument/hover(53)
I[15:30:54.598] --> reply:textDocument/hover(53) 0 ms
I[15:30:58.333] <-- textDocument/codeAction(54)
I[15:30:58.334] --> reply:textDocument/codeAction(54) 0 ms
I[15:30:58.340] <-- textDocument/documentLink(55)
I[15:30:58.340] --> reply:textDocument/documentLink(55) 0 ms
I[15:30:58.348] <-- textDocument/documentHighlight(56)
I[15:30:58.348] --> reply:textDocument/documentHighlight(56) 0 ms
I[15:30:58.355] <-- textDocument/hover(57)
I[15:30:58.357] --> reply:textDocument/hover(57) 1 ms
I[15:31:05.360] <-- textDocument/codeAction(58)
I[15:31:05.361] --> reply:textDocument/codeAction(58) 0 ms
I[15:31:05.365] <-- textDocument/documentLink(59)
I[15:31:05.365] --> reply:textDocument/documentLink(59) 0 ms
I[15:31:05.371] <-- textDocument/documentHighlight(60)
I[15:31:05.372] --> reply:textDocument/documentHighlight(60) 0 ms
I[15:31:05.381] <-- textDocument/hover(61)
I[15:31:05.383] --> reply:textDocument/hover(61) 1 ms
I[15:31:07.214] <-- textDocument/codeAction(62)
I[15:31:07.214] --> reply:textDocument/codeAction(62) 0 ms
I[15:31:07.220] <-- textDocument/documentLink(63)
I[15:31:07.220] --> reply:textDocument/documentLink(63) 0 ms
I[15:31:07.232] <-- textDocument/documentHighlight(64)
I[15:31:07.232] --> reply:textDocument/documentHighlight(64) 0 ms
I[15:31:07.233] <-- textDocument/hover(65)
I[15:31:07.234] --> reply:textDocument/hover(65) 0 ms
I[15:31:13.586] <-- textDocument/codeAction(66)
I[15:31:13.586] --> reply:textDocument/codeAction(66) 0 ms
I[15:31:13.592] <-- textDocument/documentLink(67)
I[15:31:13.592] --> reply:textDocument/documentLink(67) 0 ms
I[15:31:13.597] <-- textDocument/documentHighlight(68)
I[15:31:13.598] --> reply:textDocument/documentHighlight(68) 0 ms
I[15:31:13.606] <-- textDocument/hover(69)
I[15:31:13.608] --> reply:textDocument/hover(69) 1 ms

[-- Attachment #3: clangd-logs --]
[-- Type: application/octet-stream, Size: 14445 bytes --]

Content-Length: 2134

{"jsonrpc":"2.0","method":"initialize","params":{"processId":null,"rootPath":"/home/albinus/tmp/minicpp","clientInfo":{"name":"emacs","version":"GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.23, cairo version 1.16.0)\n of 2020-11-21"},"rootUri":"file:///home/albinus/tmp/minicpp","capabilities":{"workspace":{"workspaceEdit":{"documentChanges":true,"resourceOperations":["create","rename","delete"]},"applyEdit":true,"symbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]}},"executeCommand":{"dynamicRegistration":false},"didChangeWatchedFiles":{"dynamicRegistration":true},"workspaceFolders":true,"configuration":true},"textDocument":{"declaration":{"linkSupport":true},"definition":{"linkSupport":true},"implementation":{"linkSupport":true},"typeDefinition":{"linkSupport":true},"synchronization":{"willSave":true,"didSave":true,"willSaveWaitUntil":true},"documentSymbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"hierarchicalDocumentSymbolSupport":true},"formatting":{"dynamicRegistration":true},"rangeFormatting":{"dynamicRegistration":true},"rename":{"dynamicRegistration":true,"prepareSupport":true},"codeAction":{"dynamicRegistration":true,"isPreferredSupport":true,"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["","quickfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","source","source.organizeImports"]}}},"completion":{"completionItem":{"snippetSupport":false,"documentationFormat":["markdown"],"resolveAdditionalTextEditsSupport":true},"contextSupport":true},"signatureHelp":{"signatureInformation":{"parameterInformation":{"labelOffsetSupport":true}}},"documentLink":{"dynamicRegistration":true,"tooltipSupport":true},"hover":{"contentFormat":["markdown","plaintext"]},"foldingRange":{"dynamicRegistration":true},"callHierarchy":{"dynamicRegistration":false},"publishDiagnostics":{"relatedInformation":true,"tagSupport":{"valueSet":[1,2]},"versionSupport":true}},"window":{"workDoneProgress":true}},"initializationOptions":null,"workDoneToken":"1"},"id":13}
Content-Length: 53

{"jsonrpc":"2.0","method":"initialized","params":{}}
Content-Length: 251

{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp","languageId":"cpp","version":0,"text":"#include <iostream>\n\nint main(void)\n{\n   std::cout << \"Hello world\\n\";\n}\n"}}}
Content-Length: 242

{"jsonrpc":"2.0","method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"range":{"start":{"line":4,"character":0},"end":{"line":4,"character":0}},"context":{"diagnostics":[]}},"id":14}
Content-Length: 141

{"jsonrpc":"2.0","method":"textDocument/documentLink","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"}},"id":15}
Content-Length: 39

{"jsonrpc":"2.0","id":0,"result":null}
Content-Length: 242

{"jsonrpc":"2.0","method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"range":{"start":{"line":4,"character":0},"end":{"line":4,"character":0}},"context":{"diagnostics":[]}},"id":16}
Content-Length: 141

{"jsonrpc":"2.0","method":"textDocument/documentLink","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"}},"id":17}
Content-Length: 242

{"jsonrpc":"2.0","method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"range":{"start":{"line":4,"character":3},"end":{"line":4,"character":3}},"context":{"diagnostics":[]}},"id":18}
Content-Length: 141

{"jsonrpc":"2.0","method":"textDocument/documentLink","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"}},"id":19}
Content-Length: 182

{"jsonrpc":"2.0","method":"textDocument/documentHighlight","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":4,"character":3}},"id":20}
Content-Length: 170

{"jsonrpc":"2.0","method":"textDocument/hover","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":4,"character":3}},"id":21}
Content-Length: 242

{"jsonrpc":"2.0","method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"range":{"start":{"line":2,"character":5},"end":{"line":2,"character":5}},"context":{"diagnostics":[]}},"id":22}
Content-Length: 141

{"jsonrpc":"2.0","method":"textDocument/documentLink","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"}},"id":23}
Content-Length: 182

{"jsonrpc":"2.0","method":"textDocument/documentHighlight","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":2,"character":5}},"id":24}
Content-Length: 170

{"jsonrpc":"2.0","method":"textDocument/hover","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":2,"character":5}},"id":25}
Content-Length: 244

{"jsonrpc":"2.0","method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"range":{"start":{"line":2,"character":10},"end":{"line":2,"character":10}},"context":{"diagnostics":[]}},"id":26}
Content-Length: 141

{"jsonrpc":"2.0","method":"textDocument/documentLink","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"}},"id":27}
Content-Length: 183

{"jsonrpc":"2.0","method":"textDocument/documentHighlight","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":2,"character":10}},"id":28}
Content-Length: 171

{"jsonrpc":"2.0","method":"textDocument/hover","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":2,"character":10}},"id":29}
Content-Length: 242

{"jsonrpc":"2.0","method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"range":{"start":{"line":2,"character":0},"end":{"line":2,"character":0}},"context":{"diagnostics":[]}},"id":30}
Content-Length: 141

{"jsonrpc":"2.0","method":"textDocument/documentLink","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"}},"id":31}
Content-Length: 182

{"jsonrpc":"2.0","method":"textDocument/documentHighlight","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":2,"character":0}},"id":32}
Content-Length: 170

{"jsonrpc":"2.0","method":"textDocument/hover","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":2,"character":0}},"id":33}
Content-Length: 242

{"jsonrpc":"2.0","method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"range":{"start":{"line":4,"character":4},"end":{"line":4,"character":4}},"context":{"diagnostics":[]}},"id":34}
Content-Length: 141

{"jsonrpc":"2.0","method":"textDocument/documentLink","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"}},"id":35}
Content-Length: 182

{"jsonrpc":"2.0","method":"textDocument/documentHighlight","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":4,"character":4}},"id":36}
Content-Length: 170

{"jsonrpc":"2.0","method":"textDocument/hover","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":4,"character":4}},"id":37}
Content-Length: 244

{"jsonrpc":"2.0","method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"range":{"start":{"line":4,"character":10},"end":{"line":4,"character":10}},"context":{"diagnostics":[]}},"id":38}
Content-Length: 141

{"jsonrpc":"2.0","method":"textDocument/documentLink","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"}},"id":39}
Content-Length: 183

{"jsonrpc":"2.0","method":"textDocument/documentHighlight","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":4,"character":10}},"id":40}
Content-Length: 171

{"jsonrpc":"2.0","method":"textDocument/hover","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":4,"character":10}},"id":41}
Content-Length: 244

{"jsonrpc":"2.0","method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"range":{"start":{"line":4,"character":14},"end":{"line":4,"character":14}},"context":{"diagnostics":[]}},"id":42}
Content-Length: 141

{"jsonrpc":"2.0","method":"textDocument/documentLink","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"}},"id":43}
Content-Length: 183

{"jsonrpc":"2.0","method":"textDocument/documentHighlight","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":4,"character":14}},"id":44}
Content-Length: 171

{"jsonrpc":"2.0","method":"textDocument/hover","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":4,"character":14}},"id":45}
Content-Length: 244

{"jsonrpc":"2.0","method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"range":{"start":{"line":0,"character":14},"end":{"line":0,"character":14}},"context":{"diagnostics":[]}},"id":46}
Content-Length: 141

{"jsonrpc":"2.0","method":"textDocument/documentLink","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"}},"id":47}
Content-Length: 183

{"jsonrpc":"2.0","method":"textDocument/documentHighlight","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":0,"character":14}},"id":48}
Content-Length: 171

{"jsonrpc":"2.0","method":"textDocument/hover","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":0,"character":14}},"id":49}
Content-Length: 242

{"jsonrpc":"2.0","method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"range":{"start":{"line":0,"character":4},"end":{"line":0,"character":4}},"context":{"diagnostics":[]}},"id":50}
Content-Length: 141

{"jsonrpc":"2.0","method":"textDocument/documentLink","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"}},"id":51}
Content-Length: 182

{"jsonrpc":"2.0","method":"textDocument/documentHighlight","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":0,"character":4}},"id":52}
Content-Length: 170

{"jsonrpc":"2.0","method":"textDocument/hover","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":0,"character":4}},"id":53}
Content-Length: 242

{"jsonrpc":"2.0","method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"range":{"start":{"line":4,"character":4},"end":{"line":4,"character":4}},"context":{"diagnostics":[]}},"id":54}
Content-Length: 141

{"jsonrpc":"2.0","method":"textDocument/documentLink","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"}},"id":55}
Content-Length: 182

{"jsonrpc":"2.0","method":"textDocument/documentHighlight","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":4,"character":4}},"id":56}
Content-Length: 170

{"jsonrpc":"2.0","method":"textDocument/hover","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":4,"character":4}},"id":57}
Content-Length: 244

{"jsonrpc":"2.0","method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"range":{"start":{"line":4,"character":10},"end":{"line":4,"character":10}},"context":{"diagnostics":[]}},"id":58}
Content-Length: 141

{"jsonrpc":"2.0","method":"textDocument/documentLink","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"}},"id":59}
Content-Length: 183

{"jsonrpc":"2.0","method":"textDocument/documentHighlight","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":4,"character":10}},"id":60}
Content-Length: 171

{"jsonrpc":"2.0","method":"textDocument/hover","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":4,"character":10}},"id":61}
Content-Length: 242

{"jsonrpc":"2.0","method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"range":{"start":{"line":6,"character":0},"end":{"line":6,"character":0}},"context":{"diagnostics":[]}},"id":62}
Content-Length: 141

{"jsonrpc":"2.0","method":"textDocument/documentLink","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"}},"id":63}
Content-Length: 182

{"jsonrpc":"2.0","method":"textDocument/documentHighlight","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":6,"character":0}},"id":64}
Content-Length: 170

{"jsonrpc":"2.0","method":"textDocument/hover","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":6,"character":0}},"id":65}
Content-Length: 242

{"jsonrpc":"2.0","method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"range":{"start":{"line":2,"character":4},"end":{"line":2,"character":4}},"context":{"diagnostics":[]}},"id":66}
Content-Length: 141

{"jsonrpc":"2.0","method":"textDocument/documentLink","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"}},"id":67}
Content-Length: 182

{"jsonrpc":"2.0","method":"textDocument/documentHighlight","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":2,"character":4}},"id":68}
Content-Length: 170

{"jsonrpc":"2.0","method":"textDocument/hover","params":{"textDocument":{"uri":"file:///home/albinus/tmp/minicpp/main.cpp"},"position":{"line":2,"character":4}},"id":69}
Content-Length: 225

{"id":69,"jsonrpc":"2.0","result":{"contents":{"kind":"markdown","value":"### function `main`  \n\n---\n→ `int`  \n\n---\n```cpp\nint main()\n```"},"range":{"end":{"character":8,"line":2},"start":{"character":4,"line":2}}}}

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Martin,

>> Have you checked, if the /tmp/clangd-remote file contains any error
>> message?
>
> I have appended the (remote) /tmp/clangd-error and /tmp/clangd-logs.
>
>> When you move the cursor onto the function name or onto the
>> std::cout, do you see any output, either as a popup or in the
>> minibuffer?
>
> Yes, in the minibuffer.
>
> FTR, I'm using Emacs 28.0.50 and Tramp 2.5.0-pre, both from git
> master. I don't know whether this matters.

Short update: I have rearranged `lsp-tramp-connection', it still
works. *Plus* I see now the same error as you. I would call this
progress :-)

I still haven't a fix yet, but now it will be easier for me to debug.

>> Best Regards,
>>
>> Martin

Best regards, Michael.

RE: Tramp and conversion of \r\n into \n

[APEL Martin]

Hi Michael,

that's good to hear. I was somewhat at a loss, what to try next. Reproducing a problem in a different environment is often half of the way towards a solution ;-)

Best Regards

Martin

> -----Original Message-----
> From: Michael Albinus <michael.albinus@gmx.de>
> Sent: 15 December 2020 19:45
> To: APEL Martin <Martin.APEL@3ds.com>
> Cc: emacs-devel@gnu.org
> Subject: Re: Tramp and conversion of \r\n into \n
>
> EXTERNAL EMAIL : The sender of this email is external to 3DS. Be wary of the
> content and do not open unexpected attachments or links. If you consider
> this email as spam, you can click the following link https://spam-
> report.3ds.com/?linktext=https://www.mailcontrol.com/sr/nWcGBqEYJGbG
> X2PQPOmvUnWT5wH6X0A_f6bvjV5kRtNFj22gAcDfZa4aUu5u5Z-
> JyRb6nBgXp9cDs2jC_2iYuQ==  (no login or additional action will be
> requested).
>
>
> Michael Albinus <michael.albinus@gmx.de> writes:
>
> Hi Martin,
>
> >> Have you checked, if the /tmp/clangd-remote file contains any error
> >> message?
> >
> > I have appended the (remote) /tmp/clangd-error and /tmp/clangd-logs.
> >
> >> When you move the cursor onto the function name or onto the
> >> std::cout, do you see any output, either as a popup or in the
> >> minibuffer?
> >
> > Yes, in the minibuffer.
> >
> > FTR, I'm using Emacs 28.0.50 and Tramp 2.5.0-pre, both from git
> > master. I don't know whether this matters.
>
> Short update: I have rearranged `lsp-tramp-connection', it still works. *Plus* I
> see now the same error as you. I would call this progress :-)
>
> I still haven't a fix yet, but now it will be easier for me to debug.
>
> >> Best Regards,
> >>
> >> Martin
>
> Best regards, Michael.
This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systèmes does not accept or assume any liability or responsibility for any use of or reliance on this email.


Please be informed that your personal data are processed according to our data privacy policy as described on our website. Should you have any questions related to personal data protection, please contact 3DS Data Protection Officer at 3DS.compliance-privacy@3ds.com<mailto:3DS.compliance-privacy@3ds.com>


For other languages, go to https://www.3ds.com/terms/email-disclaimer

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

APEL Martin <Martin.APEL@3ds.com> writes:

> Hi Michael,

Hi Martin,

> 6. Now try to open the main.cpp file on the remote machine in Emacs on
> the local machine. This should trigger the start of clangd via
> lsp-mode. If the error occurs, the modeline shows 'clangd starting'
> forever.

Well, I don't belive it is eol conversion. With some more stderr
redirection, I could see that the local process sends

--8<---------------cut here---------------start------------->8---
Content-Length: 2134

{"jsonrpc":"2.0","method":"initialize","params":{"processId":null,"rootPath":"/home/albinus/tmp/minicpp","clientInfo":{"name":"emacs","version":"GNU Emacs 28.0.50 (build 28, x86_64-pc-linux-gnu, GTK+ Version 3.24.23, cairo version 1.16.0)\n of 2020-12-15"},"rootUri":"file:///home/albinus/tmp/minicpp","capabilities":{"workspace":{"workspaceEdit":{"documentChanges":true,"resourceOperations":["create","rename","delete"]},"applyEdit":true,"symbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]}},"executeCommand":{"dynamicRegistration":false},"didChangeWatchedFiles":{"dynamicRegistration":true},"workspaceFolders":true,"configuration":true},"textDocument":{"declaration":{"linkSupport":true},"definition":{"linkSupport":true},"implementation":{"linkSupport":true},"typeDefinition":{"linkSupport":true},"synchronization":{"willSave":true,"didSave":true,"willSaveWaitUntil":true},"documentSymbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"hierarchicalDocumentSymbolSupport":true},"formatting":{"dynamicRegistration":true},"rangeFormatting":{"dynamicRegistration":true},"rename":{"dynamicRegistration":true,"prepareSupport":true},"codeAction":{"dynamicRegistration":true,"isPreferredSupport":true,"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["","quickfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","source","source.organizeImports"]}}},"completion":{"completionItem":{"snippetSupport":false,"documentationFormat":["markdown"],"resolveAdditionalTextEditsSupport":true},"contextSupport":true},"signatureHelp":{"signatureInformation":{"parameterInformation":{"labelOffsetSupport":true}}},"documentLink":{"dynamicRegistration":true,"tooltipSupport":true},"hover":{"contentFormat":["markdown","plaintext"]},"foldingRange":{"dynamicRegistration":true},"callHierarchy":{"dynamicRegistration":false},"publishDiagnostics":{"relatedInformation":true,"tagSupport":{"valueSet":[1,2]},"versionSupport":true}},"window":{"workDoneProgress":true}},"initializationOptions":null,"workDoneToken":"1"},"id":1}
--8<---------------cut here---------------end--------------->8---

Both on the remote side, running "clangd --log=verbose", we see only

--8<---------------cut here---------------start------------->8---
I[16:18:53.667] Ubuntu clangd version 11.0.0-2
I[16:18:53.667] PID: 269465
I[16:18:53.667] Working directory: /home/albinus/tmp/minicpp
I[16:18:53.667] argv[0]: clangd
I[16:18:53.667] argv[1]: --log=verbose
V[16:18:53.667] User config file is /home/albinus/.config/clangd/config.yaml
I[16:18:53.667] Starting LSP over stdin/stdout
V[16:18:53.668] <<< 

{"jsonrpc":"2.0","method":"initialize","params":{"processId":null,"rootPath":"/home/albinus/tmp/minicpp","clientInfo":{"name":"emacs","version":"GNU Emacs 28.0.50 (build 28, x86_64-pc-linux-gnu, GTK+ Version 3.24.23, cairo version 1.16.0)\n of 2020-12-15"},"rootUri":"file:///home/albinus/tmp/minicpp","capabilities":{"workspace":{"workspaceEdit":{"documentChanges":true,"resourceOperations":["create","rename","delete"]},"applyEdit":true,"symbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]}},"executeCommand":{"dynamicRegistration":false},"didChangeWatchedFiles":{"dynamicRegistration":true},"workspaceFolders":true,"configuration":true},"textDocument":{"declaration":{"linkSupport":true},"definition":{"linkSupport":true},"implementation":{"linkSupport":true},"typeDefinition":{"linkSupport":true},"synchronization":{"willSave":true,"didSave":true,"willSaveWaitUntil":true},"documentSymbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"hierarchicalDocumentSymbolSupport":true},"formatting":{"dynamicRegistration":true},"rangeFormatting":{"dynamicRegistration":true},"rename":{"dynamicRegistration":true,"prepareSupport":true},"codeAction":{"dynamicRegistration":true,"isPreferredSupport":true,"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["","quickfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","source","source.organizeImports"]}}},"completion":{"completionItem":{"snippetSupport":false,"documentationFormat":["markdown"],"resolveAdditionalTextEditsSupport":true},"contextSupport":true},"signatureHelp":{"signatureInformation":{"parameterInformation":{"labelOffsetSupport":true}}},"documentLink":{"dynamicRegistration":true,"tooltipSupport":true},"hover":{"contentFormat":["markdown","plaintext"]},"foldingRange":{"dynamicRegistration":true},"callHierarchy":{"dynamicRegistration":false},"publishDiagnostics":{"relatedInformation":true,"tagSupport":{"valueSet":[1,2]},"versionSupport":true}},"window":{"workDoneProgress":true}},"initializationOptions":null,"workDoneToken":"1"},"id":1

E[16:18:53.668] JSON parse error: [3:2132, byte=2134]: Expected , or } after object property
--8<---------------cut here---------------end--------------->8---

That is, both the Content-Length header is missing as well as the final "}".
This is strange. I've tried to debug lsp-mode.el, but it is an 8200+
lines monster, and not documented well. However, lsp-tramp-connection
uses start-file-process-shell-command, which means several shell calls
in between. So I have rewritten this to the following code:

--8<---------------cut here---------------start------------->8---

(defun lsp-tramp-connection (local-command &optional generate-error-file-fn)
  "Create LSP stdio connection named name.
LOCAL-COMMAND is either list of strings, string or function which
returns the command to execute."
  ;; Force a direct asynchronous process.
  (add-to-list 'tramp-connection-properties
	       (list (regexp-quote (file-remote-p default-directory))
		     "direct-async-process" t))
  (list :connect (lambda (filter sentinel name environment-fn)
                   (let* ((final-command (lsp-resolve-final-function local-command))
			  (_stderr (or (when generate-error-file-fn
                                        (funcall generate-error-file-fn name))
                                      (format "/tmp/%s-%s-stderr" name
                                              (cl-incf lsp--stderr-index))))
                          (process-name (generate-new-buffer-name name))
                          (process-environment
                           (lsp--compute-process-environment environment-fn))
			  (proc (make-process
				 :name process-name
				 :buffer (format "*%s*" process-name)
				 :command final-command
				 :connection-type 'pipe
				 :coding 'no-conversion
				 :noquery t
				 :filter filter
				 :sentinel sentinel
				 :file-handler t)))
                     (cons proc proc)))
        :test? (lambda () (-> local-command lsp-resolve-final-function lsp-server-present?))))
--8<---------------cut here---------------end--------------->8---

With this, your test scenario works fine for me. Could you check how it
fits in your environment?

One disadvantage is, that direct asynch processes work only with the
upcoming Tramp 2.5 (that's already in Emacs master), and it works only
if the asynchronous process does not require password handling. The
latter is true, if your ssh authentication is based on keys, or if you
use Tramp's control master arguments (enabled by default).

> Hope this helps,
>
> Martin

Best regards, Michael.

RE: Tramp and conversion of \r\n into \n

[APEL Martin]

Hi Michael,

Thanks for your feedback. I think, the reason, why you only see the json contents without the header and with the last closing brace missing, is that you take the input to clangd --verbose, which is after clangd has read it. My wrapper script intercepted the input and therefore still had the header and the closing brace. Anyway your change looks promising and I will build an Emacs version from master with your change applied and let you know, if this fixes the problem.
I will also ping the guy from lsp-mode and point him to your answer, so he can possibly respond as well.

Best Regards,

Martin

> -----Original Message-----
> From: Michael Albinus <michael.albinus@gmx.de>
> Sent: 16 December 2020 16:35
> To: APEL Martin <Martin.APEL@3ds.com>
> Cc: emacs-devel@gnu.org
> Subject: Re: Tramp and conversion of \r\n into \n
>
>
> APEL Martin <Martin.APEL@3ds.com> writes:
>
> > Hi Michael,
>
> Hi Martin,
>
> > 6. Now try to open the main.cpp file on the remote machine in Emacs on
> > the local machine. This should trigger the start of clangd via
> > lsp-mode. If the error occurs, the modeline shows 'clangd starting'
> > forever.
>
> Well, I don't belive it is eol conversion. With some more stderr redirection, I
> could see that the local process sends
>
> --8<---------------cut here---------------start------------->8---
> Content-Length: 2134
>
> {"jsonrpc":"2.0","method":"initialize","params":{"processId":null,"rootPath":
> "/home/albinus/tmp/minicpp","clientInfo":{"name":"emacs","version":"GN
> U Emacs 28.0.50 (build 28, x86_64-pc-linux-gnu, GTK+ Version 3.24.23, cairo
> version 1.16.0)\n of 2020-12-
> 15"},"rootUri":"file:///home/albinus/tmp/minicpp","capabilities":{"workspac
> e":{"workspaceEdit":{"documentChanges":true,"resourceOperations":["crea
> te","rename","delete"]},"applyEdit":true,"symbol":{"symbolKind":{"valueSe
> t":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]}},"exe
> cuteCommand":{"dynamicRegistration":false},"didChangeWatchedFiles":{"d
> ynamicRegistration":true},"workspaceFolders":true,"configuration":true},"te
> xtDocument":{"declaration":{"linkSupport":true},"definition":{"linkSupport":
> true},"implementation":{"linkSupport":true},"typeDefinition":{"linkSupport":
> true},"synchronization":{"willSave":true,"didSave":true,"willSaveWaitUntil":t
> rue},"documentSymbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,1
> 2,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"hierarchicalDocumentSymbolS
> upport":true},"formatting":{"dynamicRegistration":true},"rangeFormatting":
> {"dynamicRegistration":true},"rename":{"dynamicRegistration":true,"prepar
> eSupport":true},"codeAction":{"dynamicRegistration":true,"isPreferredSupp
> ort":true,"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["","qui
> ckfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","sourc
> e","source.organizeImports"]}}},"completion":{"completionItem":{"snippetS
> upport":false,"documentationFormat":["markdown"],"resolveAdditionalTex
> tEditsSupport":true},"contextSupport":true},"signatureHelp":{"signatureInfo
> rmation":{"parameterInformation":{"labelOffsetSupport":true}}},"document
> Link":{"dynamicRegistration":true,"tooltipSupport":true},"hover":{"contentF
> ormat":["markdown","plaintext"]},"foldingRange":{"dynamicRegistration":tr
> ue},"callHierarchy":{"dynamicRegistration":false},"publishDiagnostics":{"relat
> edInformation":true,"tagSupport":{"valueSet":[1,2]},"versionSupport":true}}
> ,"window":{"workDoneProgress":true}},"initializationOptions":null,"workDo
> neToken":"1"},"id":1}
> --8<---------------cut here---------------end--------------->8---
>
> Both on the remote side, running "clangd --log=verbose", we see only
>
> --8<---------------cut here---------------start------------->8---
> I[16:18:53.667] Ubuntu clangd version 11.0.0-2 I[16:18:53.667] PID: 269465
> I[16:18:53.667] Working directory: /home/albinus/tmp/minicpp
> I[16:18:53.667] argv[0]: clangd I[16:18:53.667] argv[1]: --log=verbose
> V[16:18:53.667] User config file is /home/albinus/.config/clangd/config.yaml
> I[16:18:53.667] Starting LSP over stdin/stdout V[16:18:53.668] <<<
>
> {"jsonrpc":"2.0","method":"initialize","params":{"processId":null,"rootPath":
> "/home/albinus/tmp/minicpp","clientInfo":{"name":"emacs","version":"GN
> U Emacs 28.0.50 (build 28, x86_64-pc-linux-gnu, GTK+ Version 3.24.23, cairo
> version 1.16.0)\n of 2020-12-
> 15"},"rootUri":"file:///home/albinus/tmp/minicpp","capabilities":{"workspac
> e":{"workspaceEdit":{"documentChanges":true,"resourceOperations":["crea
> te","rename","delete"]},"applyEdit":true,"symbol":{"symbolKind":{"valueSe
> t":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]}},"exe
> cuteCommand":{"dynamicRegistration":false},"didChangeWatchedFiles":{"d
> ynamicRegistration":true},"workspaceFolders":true,"configuration":true},"te
> xtDocument":{"declaration":{"linkSupport":true},"definition":{"linkSupport":
> true},"implementation":{"linkSupport":true},"typeDefinition":{"linkSupport":
> true},"synchronization":{"willSave":true,"didSave":true,"willSaveWaitUntil":t
> rue},"documentSymbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,1
> 2,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"hierarchicalDocumentSymbolS
> upport":true},"formatting":{"dynamicRegistration":true},"rangeFormatting":
> {"dynamicRegistration":true},"rename":{"dynamicRegistration":true,"prepar
> eSupport":true},"codeAction":{"dynamicRegistration":true,"isPreferredSupp
> ort":true,"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["","qui
> ckfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","sourc
> e","source.organizeImports"]}}},"completion":{"completionItem":{"snippetS
> upport":false,"documentationFormat":["markdown"],"resolveAdditionalTex
> tEditsSupport":true},"contextSupport":true},"signatureHelp":{"signatureInfo
> rmation":{"parameterInformation":{"labelOffsetSupport":true}}},"document
> Link":{"dynamicRegistration":true,"tooltipSupport":true},"hover":{"contentF
> ormat":["markdown","plaintext"]},"foldingRange":{"dynamicRegistration":tr
> ue},"callHierarchy":{"dynamicRegistration":false},"publishDiagnostics":{"relat
> edInformation":true,"tagSupport":{"valueSet":[1,2]},"versionSupport":true}}
> ,"window":{"workDoneProgress":true}},"initializationOptions":null,"workDo
> neToken":"1"},"id":1
>
> E[16:18:53.668] JSON parse error: [3:2132, byte=2134]: Expected , or } after
> object property --8<---------------cut here---------------end--------------->8---
>
> That is, both the Content-Length header is missing as well as the final "}".
> This is strange. I've tried to debug lsp-mode.el, but it is an 8200+ lines
> monster, and not documented well. However, lsp-tramp-connection uses
> start-file-process-shell-command, which means several shell calls in
> between. So I have rewritten this to the following code:
>
> --8<---------------cut here---------------start------------->8---
>
> (defun lsp-tramp-connection (local-command &optional generate-error-file-
> fn)
>   "Create LSP stdio connection named name.
> LOCAL-COMMAND is either list of strings, string or function which returns the
> command to execute."
>   ;; Force a direct asynchronous process.
>   (add-to-list 'tramp-connection-properties
>        (list (regexp-quote (file-remote-p default-directory))
>      "direct-async-process" t))
>   (list :connect (lambda (filter sentinel name environment-fn)
>                    (let* ((final-command (lsp-resolve-final-function local-command))
>   (_stderr (or (when generate-error-file-fn
>                                         (funcall generate-error-file-fn name))
>                                       (format "/tmp/%s-%s-stderr" name
>                                               (cl-incf lsp--stderr-index))))
>                           (process-name (generate-new-buffer-name name))
>                           (process-environment
>                            (lsp--compute-process-environment environment-fn))
>   (proc (make-process
>  :name process-name
>  :buffer (format "*%s*" process-name)
>  :command final-command
>  :connection-type 'pipe
>  :coding 'no-conversion
>  :noquery t
>  :filter filter
>  :sentinel sentinel
>  :file-handler t)))
>                      (cons proc proc)))
>         :test? (lambda () (-> local-command lsp-resolve-final-function lsp-server-
> present?)))) --8<---------------cut here---------------end--------------->8---
>
> With this, your test scenario works fine for me. Could you check how it fits in
> your environment?
>
> One disadvantage is, that direct asynch processes work only with the
> upcoming Tramp 2.5 (that's already in Emacs master), and it works only if the
> asynchronous process does not require password handling. The latter is true,
> if your ssh authentication is based on keys, or if you use Tramp's control
> master arguments (enabled by default).
>
> > Hope this helps,
> >
> > Martin
>
> Best regards, Michael.
This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systèmes does not accept or assume any liability or responsibility for any use of or reliance on this email.


Please be informed that your personal data are processed according to our data privacy policy as described on our website. Should you have any questions related to personal data protection, please contact 3DS Data Protection Officer at 3DS.compliance-privacy@3ds.com<mailto:3DS.compliance-privacy@3ds.com>


For other languages, go to https://www.3ds.com/terms/email-disclaimer

RE: Tramp and conversion of \r\n into \n

[APEL Martin]

Hi Michael,

I built an Emacs version from the current master branch and tried it out:
lsp-mode works well with a remote clangd now. Unfortunately there are some other problems, which do not occur with Emacs 27.1 and which might be related to TRAMP:
- my remote compile setup does not work anymore. It seems that the arguments to the invoked compile command are lost on the way to the remote machine
- rgrep on a remote machine also does not work. It produces a list of files it intends to search instead of the grep output
- I also encountered an error message from TRAMP once. Unfortunately I restarted Emacs afterwards, so I don't have the exact error message. It was something about a recursive call detected and a sentinel was mentioned.

Best Regards,

Martin

> -----Original Message-----
> From: Michael Albinus <michael.albinus@gmx.de>
> Sent: 16 December 2020 16:35
> To: APEL Martin <Martin.APEL@3ds.com>
> Cc: emacs-devel@gnu.org
> Subject: Re: Tramp and conversion of \r\n into \n
>
>
> APEL Martin <Martin.APEL@3ds.com> writes:
>
> > Hi Michael,
>
> Hi Martin,
>
> > 6. Now try to open the main.cpp file on the remote machine in Emacs on
> > the local machine. This should trigger the start of clangd via
> > lsp-mode. If the error occurs, the modeline shows 'clangd starting'
> > forever.
>
> Well, I don't belive it is eol conversion. With some more stderr redirection, I
> could see that the local process sends
>
> --8<---------------cut here---------------start------------->8---
> Content-Length: 2134
>
> {"jsonrpc":"2.0","method":"initialize","params":{"processId":null,"rootPath":
> "/home/albinus/tmp/minicpp","clientInfo":{"name":"emacs","version":"GN
> U Emacs 28.0.50 (build 28, x86_64-pc-linux-gnu, GTK+ Version 3.24.23, cairo
> version 1.16.0)\n of 2020-12-
> 15"},"rootUri":"file:///home/albinus/tmp/minicpp","capabilities":{"workspac
> e":{"workspaceEdit":{"documentChanges":true,"resourceOperations":["crea
> te","rename","delete"]},"applyEdit":true,"symbol":{"symbolKind":{"valueSe
> t":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]}},"exe
> cuteCommand":{"dynamicRegistration":false},"didChangeWatchedFiles":{"d
> ynamicRegistration":true},"workspaceFolders":true,"configuration":true},"te
> xtDocument":{"declaration":{"linkSupport":true},"definition":{"linkSupport":
> true},"implementation":{"linkSupport":true},"typeDefinition":{"linkSupport":
> true},"synchronization":{"willSave":true,"didSave":true,"willSaveWaitUntil":t
> rue},"documentSymbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,1
> 2,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"hierarchicalDocumentSymbolS
> upport":true},"formatting":{"dynamicRegistration":true},"rangeFormatting":
> {"dynamicRegistration":true},"rename":{"dynamicRegistration":true,"prepar
> eSupport":true},"codeAction":{"dynamicRegistration":true,"isPreferredSupp
> ort":true,"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["","qui
> ckfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","sourc
> e","source.organizeImports"]}}},"completion":{"completionItem":{"snippetS
> upport":false,"documentationFormat":["markdown"],"resolveAdditionalTex
> tEditsSupport":true},"contextSupport":true},"signatureHelp":{"signatureInfo
> rmation":{"parameterInformation":{"labelOffsetSupport":true}}},"document
> Link":{"dynamicRegistration":true,"tooltipSupport":true},"hover":{"contentF
> ormat":["markdown","plaintext"]},"foldingRange":{"dynamicRegistration":tr
> ue},"callHierarchy":{"dynamicRegistration":false},"publishDiagnostics":{"relat
> edInformation":true,"tagSupport":{"valueSet":[1,2]},"versionSupport":true}}
> ,"window":{"workDoneProgress":true}},"initializationOptions":null,"workDo
> neToken":"1"},"id":1}
> --8<---------------cut here---------------end--------------->8---
>
> Both on the remote side, running "clangd --log=verbose", we see only
>
> --8<---------------cut here---------------start------------->8---
> I[16:18:53.667] Ubuntu clangd version 11.0.0-2 I[16:18:53.667] PID: 269465
> I[16:18:53.667] Working directory: /home/albinus/tmp/minicpp
> I[16:18:53.667] argv[0]: clangd I[16:18:53.667] argv[1]: --log=verbose
> V[16:18:53.667] User config file is /home/albinus/.config/clangd/config.yaml
> I[16:18:53.667] Starting LSP over stdin/stdout V[16:18:53.668] <<<
>
> {"jsonrpc":"2.0","method":"initialize","params":{"processId":null,"rootPath":
> "/home/albinus/tmp/minicpp","clientInfo":{"name":"emacs","version":"GN
> U Emacs 28.0.50 (build 28, x86_64-pc-linux-gnu, GTK+ Version 3.24.23, cairo
> version 1.16.0)\n of 2020-12-
> 15"},"rootUri":"file:///home/albinus/tmp/minicpp","capabilities":{"workspac
> e":{"workspaceEdit":{"documentChanges":true,"resourceOperations":["crea
> te","rename","delete"]},"applyEdit":true,"symbol":{"symbolKind":{"valueSe
> t":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]}},"exe
> cuteCommand":{"dynamicRegistration":false},"didChangeWatchedFiles":{"d
> ynamicRegistration":true},"workspaceFolders":true,"configuration":true},"te
> xtDocument":{"declaration":{"linkSupport":true},"definition":{"linkSupport":
> true},"implementation":{"linkSupport":true},"typeDefinition":{"linkSupport":
> true},"synchronization":{"willSave":true,"didSave":true,"willSaveWaitUntil":t
> rue},"documentSymbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,1
> 2,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"hierarchicalDocumentSymbolS
> upport":true},"formatting":{"dynamicRegistration":true},"rangeFormatting":
> {"dynamicRegistration":true},"rename":{"dynamicRegistration":true,"prepar
> eSupport":true},"codeAction":{"dynamicRegistration":true,"isPreferredSupp
> ort":true,"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["","qui
> ckfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","sourc
> e","source.organizeImports"]}}},"completion":{"completionItem":{"snippetS
> upport":false,"documentationFormat":["markdown"],"resolveAdditionalTex
> tEditsSupport":true},"contextSupport":true},"signatureHelp":{"signatureInfo
> rmation":{"parameterInformation":{"labelOffsetSupport":true}}},"document
> Link":{"dynamicRegistration":true,"tooltipSupport":true},"hover":{"contentF
> ormat":["markdown","plaintext"]},"foldingRange":{"dynamicRegistration":tr
> ue},"callHierarchy":{"dynamicRegistration":false},"publishDiagnostics":{"relat
> edInformation":true,"tagSupport":{"valueSet":[1,2]},"versionSupport":true}}
> ,"window":{"workDoneProgress":true}},"initializationOptions":null,"workDo
> neToken":"1"},"id":1
>
> E[16:18:53.668] JSON parse error: [3:2132, byte=2134]: Expected , or } after
> object property --8<---------------cut here---------------end--------------->8---
>
> That is, both the Content-Length header is missing as well as the final "}".
> This is strange. I've tried to debug lsp-mode.el, but it is an 8200+ lines
> monster, and not documented well. However, lsp-tramp-connection uses
> start-file-process-shell-command, which means several shell calls in
> between. So I have rewritten this to the following code:
>
> --8<---------------cut here---------------start------------->8---
>
> (defun lsp-tramp-connection (local-command &optional generate-error-file-
> fn)
>   "Create LSP stdio connection named name.
> LOCAL-COMMAND is either list of strings, string or function which returns the
> command to execute."
>   ;; Force a direct asynchronous process.
>   (add-to-list 'tramp-connection-properties
>        (list (regexp-quote (file-remote-p default-directory))
>      "direct-async-process" t))
>   (list :connect (lambda (filter sentinel name environment-fn)
>                    (let* ((final-command (lsp-resolve-final-function local-command))
>   (_stderr (or (when generate-error-file-fn
>                                         (funcall generate-error-file-fn name))
>                                       (format "/tmp/%s-%s-stderr" name
>                                               (cl-incf lsp--stderr-index))))
>                           (process-name (generate-new-buffer-name name))
>                           (process-environment
>                            (lsp--compute-process-environment environment-fn))
>   (proc (make-process
>  :name process-name
>  :buffer (format "*%s*" process-name)
>  :command final-command
>  :connection-type 'pipe
>  :coding 'no-conversion
>  :noquery t
>  :filter filter
>  :sentinel sentinel
>  :file-handler t)))
>                      (cons proc proc)))
>         :test? (lambda () (-> local-command lsp-resolve-final-function lsp-server-
> present?)))) --8<---------------cut here---------------end--------------->8---
>
> With this, your test scenario works fine for me. Could you check how it fits in
> your environment?
>
> One disadvantage is, that direct asynch processes work only with the
> upcoming Tramp 2.5 (that's already in Emacs master), and it works only if the
> asynchronous process does not require password handling. The latter is true,
> if your ssh authentication is based on keys, or if you use Tramp's control
> master arguments (enabled by default).
>
> > Hope this helps,
> >
> > Martin
>
> Best regards, Michael.
This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systèmes does not accept or assume any liability or responsibility for any use of or reliance on this email.


Please be informed that your personal data are processed according to our data privacy policy as described on our website. Should you have any questions related to personal data protection, please contact 3DS Data Protection Officer at 3DS.compliance-privacy@3ds.com<mailto:3DS.compliance-privacy@3ds.com>


For other languages, go to https://www.3ds.com/terms/email-disclaimer

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

APEL Martin <Martin.APEL@3ds.com> writes:

> Hi Michael,

Hi Martin,

> I built an Emacs version from the current master branch and tried it out:
> lsp-mode works well with a remote clangd now.

Thanks for the feedback.

> Unfortunately there are
> some other problems, which do not occur with Emacs 27.1 and which
> might be related to TRAMP:
> - my remote compile setup does not work anymore. It seems that the
> arguments to the invoked compile command are lost on the way to the
> remote machine
> - rgrep on a remote machine also does not work. It produces a list of
> files it intends to search instead of the grep output

Direct async calls are new in Emacs 28, so they might not work yet in
all cases. Pls run your test with tramp-verbose set to 10, and send me
the debug traces in both cases. It would also good to know, how compile
and rgrep are invoked.

> - I also encountered an error message from TRAMP once. Unfortunately I
> restarted Emacs afterwards, so I don't have the exact error
> message. It was something about a recursive call detected and a
> sentinel was mentioned.

This is not related to the lsp-mode change, but rather to hardening
Tramp detecting blocking situations (you've seen my message about "Tramp
and timers"). Process sentinels are also affected.

In order to circumvent this, you might add to your .emacs

(setq debug-ignored-errors (cons 'remote-file-error debug-ignored-errors))

> Best Regards,
>
> Martin

Best regards, Michael.

RE: Tramp and conversion of \r\n into \n

[APEL Martin]

[-- Attachment #1: Type: text/plain, Size: 3334 bytes --]

Hi Michael,

I have created the trace files for remote compile and rgrep for you and attached them.
For the remote compile I invoke a shell script called 'my_remote_compile.sh', which in turn calls a script 'my_compile.sh'. The script is called with two arguments, the build directory and the target name, which are lost, when the script is actually called. This works in Emacs 27.1.
For the rgrep invocation I simply used a pattern 'mytest' and '*' for the file pattern.

Regarding the error message about recursive invocation, I have not seen this anymore.

Best Regards,

Martin

> -----Original Message-----
> From: Michael Albinus <michael.albinus@gmx.de>
> Sent: 17 December 2020 13:50
> To: APEL Martin <Martin.APEL@3ds.com>
> Cc: emacs-devel@gnu.org
> Subject: Re: Tramp and conversion of \r\n into \n
>
>
> APEL Martin <Martin.APEL@3ds.com> writes:
>
> > Hi Michael,
>
> Hi Martin,
>
> > I built an Emacs version from the current master branch and tried it out:
> > lsp-mode works well with a remote clangd now.
>
> Thanks for the feedback.
>
> > Unfortunately there are
> > some other problems, which do not occur with Emacs 27.1 and which
> > might be related to TRAMP:
> > - my remote compile setup does not work anymore. It seems that the
> > arguments to the invoked compile command are lost on the way to the
> > remote machine
> > - rgrep on a remote machine also does not work. It produces a list of
> > files it intends to search instead of the grep output
>
> Direct async calls are new in Emacs 28, so they might not work yet in all cases.
> Pls run your test with tramp-verbose set to 10, and send me the debug
> traces in both cases. It would also good to know, how compile and rgrep are
> invoked.
>
> > - I also encountered an error message from TRAMP once. Unfortunately I
> > restarted Emacs afterwards, so I don't have the exact error message.
> > It was something about a recursive call detected and a sentinel was
> > mentioned.
>
> This is not related to the lsp-mode change, but rather to hardening Tramp
> detecting blocking situations (you've seen my message about "Tramp and
> timers"). Process sentinels are also affected.
>
> In order to circumvent this, you might add to your .emacs
>
> (setq debug-ignored-errors (cons 'remote-file-error debug-ignored-errors))
>
> > Best Regards,
> >
> > Martin
>
> Best regards, Michael.
This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systèmes does not accept or assume any liability or responsibility for any use of or reliance on this email.


Please be informed that your personal data are processed according to our data privacy policy as described on our website. Should you have any questions related to personal data protection, please contact 3DS Data Protection Officer at 3DS.compliance-privacy@3ds.com<mailto:3DS.compliance-privacy@3ds.com>


For other languages, go to https://www.3ds.com/terms/email-disclaimer

[-- Attachment #2: remote_compile.trace.gz --]
[-- Type: application/x-gzip, Size: 115146 bytes --]

[-- Attachment #3: remote_grep.trace.gz --]
[-- Type: application/x-gzip, Size: 67389 bytes --]

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

APEL Martin <Martin.APEL@3ds.com> writes:

> Hi Michael,

Hi Martin,

> I have created the trace files for remote compile and rgrep for you
> and attached them.
> For the remote compile I invoke a shell script called
> 'my_remote_compile.sh', which in turn calls a script
> 'my_compile.sh'. The script is called with two arguments, the build
> directory and the target name, which are lost, when the script is
> actually called. This works in Emacs 27.1.
> For the rgrep invocation I simply used a pattern 'mytest' and '*' for
> the file pattern.

Thanks. I've pushed a fix to Emacs master, could you pls check?

> Best Regards,
>
> Martin

Best regards, Michael.

RE: Tramp and conversion of \r\n into \n

[APEL Martin]

Hi Michael,

With the most recent master version both remote compile and rgrep work again. And lsp-mode also still works! So this seems to be a solution for people living on the bleeding edge.

Thanks for your help,

Martin

> -----Original Message-----
> From: Michael Albinus <michael.albinus@gmx.de>
> Sent: 18 December 2020 13:36
> To: APEL Martin <Martin.APEL@3ds.com>
> Cc: emacs-devel@gnu.org
> Subject: Re: Tramp and conversion of \r\n into \n
>
>
> APEL Martin <Martin.APEL@3ds.com> writes:
>
> > Hi Michael,
>
> Hi Martin,
>
> > I have created the trace files for remote compile and rgrep for you
> > and attached them.
> > For the remote compile I invoke a shell script called
> > 'my_remote_compile.sh', which in turn calls a script 'my_compile.sh'.
> > The script is called with two arguments, the build directory and the
> > target name, which are lost, when the script is actually called. This
> > works in Emacs 27.1.
> > For the rgrep invocation I simply used a pattern 'mytest' and '*' for
> > the file pattern.
>
> Thanks. I've pushed a fix to Emacs master, could you pls check?
>
> > Best Regards,
> >
> > Martin
>
> Best regards, Michael.
This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systèmes does not accept or assume any liability or responsibility for any use of or reliance on this email.


Please be informed that your personal data are processed according to our data privacy policy as described on our website. Should you have any questions related to personal data protection, please contact 3DS Data Protection Officer at 3DS.compliance-privacy@3ds.com<mailto:3DS.compliance-privacy@3ds.com>


For other languages, go to https://www.3ds.com/terms/email-disclaimer

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

APEL Martin <Martin.APEL@3ds.com> writes:

> Hi Michael,

Hi Martin,

> With the most recent master version both remote compile and rgrep work
> again. And lsp-mode also still works! So this seems to be a solution
> for people living on the bleeding edge.

Thanks for the feedback, this are good news.

I will continue to work on this part of Tramp. Two recently added test
cases in tramp-tests.el do not work sufficiently yet for direct async
processes (using environment variables), but this shouldn't stop you to
use the current code in lsp-mode.

> Thanks for your help,
>
> Martin

Best regards, Michael.

RE: Tramp and conversion of \r\n into \n

[APEL Martin]

Hi Michael,

I just found one remaining issue: when trying to commit in a remote repository via magit, this operation fails. The magit-process window contains the message 'env: -c: No such file or directory' afterwards. Unfortunately I cannot send you tramp logs this time, as this would contain parts of the source code I am working on, which I'm not allowed to pass. I hope you can find the issue nevertheless, you probably use magit yourself.

Best Regards,

Martin
> -----Original Message-----
> From: Michael Albinus <michael.albinus@gmx.de>
> Sent: 18 December 2020 14:05
> To: APEL Martin <Martin.APEL@3ds.com>
> Cc: emacs-devel@gnu.org
> Subject: Re: Tramp and conversion of \r\n into \n
>
> APEL Martin <Martin.APEL@3ds.com> writes:
>
> > Hi Michael,
>
> Hi Martin,
>
> > With the most recent master version both remote compile and rgrep work
> > again. And lsp-mode also still works! So this seems to be a solution
> > for people living on the bleeding edge.
>
> Thanks for the feedback, this are good news.
>
> I will continue to work on this part of Tramp. Two recently added test cases in
> tramp-tests.el do not work sufficiently yet for direct async processes (using
> environment variables), but this shouldn't stop you to use the current code
> in lsp-mode.
>
> > Thanks for your help,
> >
> > Martin
>
> Best regards, Michael.
This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systèmes does not accept or assume any liability or responsibility for any use of or reliance on this email.


Please be informed that your personal data are processed according to our data privacy policy as described on our website. Should you have any questions related to personal data protection, please contact 3DS Data Protection Officer at 3DS.compliance-privacy@3ds.com<mailto:3DS.compliance-privacy@3ds.com>


For other languages, go to https://www.3ds.com/terms/email-disclaimer

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

APEL Martin <Martin.APEL@3ds.com> writes:

> Hi Michael,

Hi Martin,

> I just found one remaining issue: when trying to commit in a remote
> repository via magit, this operation fails. The magit-process window
> contains the message 'env: -c: No such file or directory'
> afterwards. Unfortunately I cannot send you tramp logs this time, as
> this would contain parts of the source code I am working on, which I'm
> not allowed to pass. I hope you can find the issue nevertheless, you
> probably use magit yourself.

This might be related to the remaining issue I'm working on, passing
environment variables.

No, I don't use magit. But you don't need to send the whole traces. Just
run the test with tramp-verbose set to at least 6. In the traces, you'll
find something like

<time stamp> tramp-handle-make-process (6) # <whole command>

Please send this line; I'm hoping <whole command> doesn't contain
sensitive data.

> Best Regards,
>
> Martin

Best regards, Michael.

RE: Tramp and conversion of \r\n into \n

[APEL Martin]

Hi Michael,

I have found two places, where tramp-handle-make-process appears:

15:48:13.218370 tramp-handle-make-process (6) # ssh -q -l mal1 -e none dell248cem cd /scratch/apel/new_arch/ && env GIT_EDITOR=sh -c 'printf "WITH-EDITOR: $$ OPEN $0\037 IN $(pwd)\n"; sleep 604800 & sleep=$!; trap "kill $sleep; exit 0" USR1; trap "kill $sleep; exit 1" USR2; wait $sleep' git --no-pager --literal-pathspecs -c core.preloadindex=true -c log.showSignature=false -c color.ui=false -c color.diff=false commit --
15:57:44.334489 tramp-handle-make-process (6) # ssh -q -l mal1 -e none dell248cem cd /scratch/apel/new_arch/ && git --no-pager --literal-pathspecs -c core.preloadindex=true -c log.showSignature=false -c color.ui=false -c color.diff=false push -v origin branchname:refs/heads/branchname

I have replaced the actual branchname by 'branchname' above.

This is my last act at work for this year, so don't expect any reply soon ;-)

Best Regards,

Martin
> -----Original Message-----
> From: Michael Albinus <michael.albinus@gmx.de>
> Sent: 18 December 2020 16:01
> To: APEL Martin <Martin.APEL@3ds.com>
> Cc: emacs-devel@gnu.org
> Subject: Re: Tramp and conversion of \r\n into \n
>
>
>
> APEL Martin <Martin.APEL@3ds.com> writes:
>
> > Hi Michael,
>
> Hi Martin,
>
> > I just found one remaining issue: when trying to commit in a remote
> > repository via magit, this operation fails. The magit-process window
> > contains the message 'env: -c: No such file or directory'
> > afterwards. Unfortunately I cannot send you tramp logs this time, as
> > this would contain parts of the source code I am working on, which I'm
> > not allowed to pass. I hope you can find the issue nevertheless, you
> > probably use magit yourself.
>
> This might be related to the remaining issue I'm working on, passing
> environment variables.
>
> No, I don't use magit. But you don't need to send the whole traces. Just run
> the test with tramp-verbose set to at least 6. In the traces, you'll find
> something like
>
> <time stamp> tramp-handle-make-process (6) # <whole command>
>
> Please send this line; I'm hoping <whole command> doesn't contain sensitive
> data.
>
> > Best Regards,
> >
> > Martin
>
> Best regards, Michael.
This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systèmes does not accept or assume any liability or responsibility for any use of or reliance on this email.


Please be informed that your personal data are processed according to our data privacy policy as described on our website. Should you have any questions related to personal data protection, please contact 3DS Data Protection Officer at 3DS.compliance-privacy@3ds.com<mailto:3DS.compliance-privacy@3ds.com>


For other languages, go to https://www.3ds.com/terms/email-disclaimer

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

APEL Martin <Martin.APEL@3ds.com> writes:

> Hi Michael,

Hi Martin,

> I have found two places, where tramp-handle-make-process appears:
>
> 15:48:13.218370 tramp-handle-make-process (6) # ssh -q -l mal1 -e none
> dell248cem cd /scratch/apel/new_arch/ && env GIT_EDITOR=sh -c 'printf
> "WITH-EDITOR: $$ OPEN $0\037 IN $(pwd)\n"; sleep 604800 & sleep=$!;
> trap "kill $sleep; exit 0" USR1; trap "kill $sleep; exit 1" USR2; wait
> $sleep' git --no-pager --literal-pathspecs -c core.preloadindex=true
> -c log.showSignature=false -c color.ui=false -c color.diff=false
> commit --
> 15:57:44.334489 tramp-handle-make-process (6) # ssh -q -l mal1 -e none
> dell248cem cd /scratch/apel/new_arch/ && git --no-pager
> --literal-pathspecs -c core.preloadindex=true -c
> log.showSignature=false -c color.ui=false -c color.diff=false push -v
> origin branchname:refs/heads/branchname
>
> I have replaced the actual branchname by 'branchname' above.

Thanks. This is a somehow different problem as the one I'm currently
working on.

> This is my last act at work for this year, so don't expect any reply soon ;-)

Merry Xmas for you and your family, and I hope you'll do well! Stay healthy!

I have cancelled all family visits until end of pandemic, so it will be
quiet here. But you gave me something to look on :-)

> Best Regards,
>
> Martin

Best regards, Michael.

Re: Tramp and conversion of \r\n into \n

[yyoncho]

[-- Attachment #1: Type: text/plain, Size: 11113 bytes --]

Michael,

About the original issue: is it possible to avoid the issue using emacs
27.1(or older)? Asking the users to move to master branch is not very
desirable since lsp-mode is used also by a lot of complete beginners.

Thanks,
Ivan

On Thu, Dec 17, 2020 at 1:48 PM APEL Martin <Martin.APEL@3ds.com> wrote:

> Hi Michael,
>
> I built an Emacs version from the current master branch and tried it out:
> lsp-mode works well with a remote clangd now. Unfortunately there are some
> other problems, which do not occur with Emacs 27.1 and which might be
> related to TRAMP:
> - my remote compile setup does not work anymore. It seems that the
> arguments to the invoked compile command are lost on the way to the remote
> machine
> - rgrep on a remote machine also does not work. It produces a list of
> files it intends to search instead of the grep output
> - I also encountered an error message from TRAMP once. Unfortunately I
> restarted Emacs afterwards, so I don't have the exact error message. It was
> something about a recursive call detected and a sentinel was mentioned.
>
> Best Regards,
>
> Martin
>
> > -----Original Message-----
> > From: Michael Albinus <michael.albinus@gmx.de>
> > Sent: 16 December 2020 16:35
> > To: APEL Martin <Martin.APEL@3ds.com>
> > Cc: emacs-devel@gnu.org
> > Subject: Re: Tramp and conversion of \r\n into \n
> >
> >
> > APEL Martin <Martin.APEL@3ds.com> writes:
> >
> > > Hi Michael,
> >
> > Hi Martin,
> >
> > > 6. Now try to open the main.cpp file on the remote machine in Emacs on
> > > the local machine. This should trigger the start of clangd via
> > > lsp-mode. If the error occurs, the modeline shows 'clangd starting'
> > > forever.
> >
> > Well, I don't belive it is eol conversion. With some more stderr
> redirection, I
> > could see that the local process sends
> >
> > --8<---------------cut here---------------start------------->8---
> > Content-Length: 2134
> >
> >
> {"jsonrpc":"2.0","method":"initialize","params":{"processId":null,"rootPath":
> > "/home/albinus/tmp/minicpp","clientInfo":{"name":"emacs","version":"GN
> > U Emacs 28.0.50 (build 28, x86_64-pc-linux-gnu, GTK+ Version 3.24.23,
> cairo
> > version 1.16.0)\n of 2020-12-
> >
> 15"},"rootUri":"file:///home/albinus/tmp/minicpp","capabilities":{"workspac
> > e":{"workspaceEdit":{"documentChanges":true,"resourceOperations":["crea
> > te","rename","delete"]},"applyEdit":true,"symbol":{"symbolKind":{"valueSe
> >
> t":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]}},"exe
> > cuteCommand":{"dynamicRegistration":false},"didChangeWatchedFiles":{"d
> >
> ynamicRegistration":true},"workspaceFolders":true,"configuration":true},"te
> >
> xtDocument":{"declaration":{"linkSupport":true},"definition":{"linkSupport":
> >
> true},"implementation":{"linkSupport":true},"typeDefinition":{"linkSupport":
> >
> true},"synchronization":{"willSave":true,"didSave":true,"willSaveWaitUntil":t
> >
> rue},"documentSymbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,1
> >
> 2,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"hierarchicalDocumentSymbolS
> >
> upport":true},"formatting":{"dynamicRegistration":true},"rangeFormatting":
> > {"dynamicRegistration":true},"rename":{"dynamicRegistration":true,"prepar
> > eSupport":true},"codeAction":{"dynamicRegistration":true,"isPreferredSupp
> >
> ort":true,"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["","qui
> >
> ckfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","sourc
> > e","source.organizeImports"]}}},"completion":{"completionItem":{"snippetS
> > upport":false,"documentationFormat":["markdown"],"resolveAdditionalTex
> >
> tEditsSupport":true},"contextSupport":true},"signatureHelp":{"signatureInfo
> > rmation":{"parameterInformation":{"labelOffsetSupport":true}}},"document
> >
> Link":{"dynamicRegistration":true,"tooltipSupport":true},"hover":{"contentF
> > ormat":["markdown","plaintext"]},"foldingRange":{"dynamicRegistration":tr
> >
> ue},"callHierarchy":{"dynamicRegistration":false},"publishDiagnostics":{"relat
> >
> edInformation":true,"tagSupport":{"valueSet":[1,2]},"versionSupport":true}}
> > ,"window":{"workDoneProgress":true}},"initializationOptions":null,"workDo
> > neToken":"1"},"id":1}
> > --8<---------------cut here---------------end--------------->8---
> >
> > Both on the remote side, running "clangd --log=verbose", we see only
> >
> > --8<---------------cut here---------------start------------->8---
> > I[16:18:53.667] Ubuntu clangd version 11.0.0-2 I[16:18:53.667] PID:
> 269465
> > I[16:18:53.667] Working directory: /home/albinus/tmp/minicpp
> > I[16:18:53.667] argv[0]: clangd I[16:18:53.667] argv[1]: --log=verbose
> > V[16:18:53.667] User config file is
> /home/albinus/.config/clangd/config.yaml
> > I[16:18:53.667] Starting LSP over stdin/stdout V[16:18:53.668] <<<
> >
> >
> {"jsonrpc":"2.0","method":"initialize","params":{"processId":null,"rootPath":
> > "/home/albinus/tmp/minicpp","clientInfo":{"name":"emacs","version":"GN
> > U Emacs 28.0.50 (build 28, x86_64-pc-linux-gnu, GTK+ Version 3.24.23,
> cairo
> > version 1.16.0)\n of 2020-12-
> >
> 15"},"rootUri":"file:///home/albinus/tmp/minicpp","capabilities":{"workspac
> > e":{"workspaceEdit":{"documentChanges":true,"resourceOperations":["crea
> > te","rename","delete"]},"applyEdit":true,"symbol":{"symbolKind":{"valueSe
> >
> t":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]}},"exe
> > cuteCommand":{"dynamicRegistration":false},"didChangeWatchedFiles":{"d
> >
> ynamicRegistration":true},"workspaceFolders":true,"configuration":true},"te
> >
> xtDocument":{"declaration":{"linkSupport":true},"definition":{"linkSupport":
> >
> true},"implementation":{"linkSupport":true},"typeDefinition":{"linkSupport":
> >
> true},"synchronization":{"willSave":true,"didSave":true,"willSaveWaitUntil":t
> >
> rue},"documentSymbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,1
> >
> 2,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"hierarchicalDocumentSymbolS
> >
> upport":true},"formatting":{"dynamicRegistration":true},"rangeFormatting":
> > {"dynamicRegistration":true},"rename":{"dynamicRegistration":true,"prepar
> > eSupport":true},"codeAction":{"dynamicRegistration":true,"isPreferredSupp
> >
> ort":true,"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["","qui
> >
> ckfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","sourc
> > e","source.organizeImports"]}}},"completion":{"completionItem":{"snippetS
> > upport":false,"documentationFormat":["markdown"],"resolveAdditionalTex
> >
> tEditsSupport":true},"contextSupport":true},"signatureHelp":{"signatureInfo
> > rmation":{"parameterInformation":{"labelOffsetSupport":true}}},"document
> >
> Link":{"dynamicRegistration":true,"tooltipSupport":true},"hover":{"contentF
> > ormat":["markdown","plaintext"]},"foldingRange":{"dynamicRegistration":tr
> >
> ue},"callHierarchy":{"dynamicRegistration":false},"publishDiagnostics":{"relat
> >
> edInformation":true,"tagSupport":{"valueSet":[1,2]},"versionSupport":true}}
> > ,"window":{"workDoneProgress":true}},"initializationOptions":null,"workDo
> > neToken":"1"},"id":1
> >
> > E[16:18:53.668] JSON parse error: [3:2132, byte=2134]: Expected , or }
> after
> > object property --8<---------------cut
> here---------------end--------------->8---
> >
> > That is, both the Content-Length header is missing as well as the final
> "}".
> > This is strange. I've tried to debug lsp-mode.el, but it is an 8200+
> lines
> > monster, and not documented well. However, lsp-tramp-connection uses
> > start-file-process-shell-command, which means several shell calls in
> > between. So I have rewritten this to the following code:
> >
> > --8<---------------cut here---------------start------------->8---
> >
> > (defun lsp-tramp-connection (local-command &optional generate-error-file-
> > fn)
> >   "Create LSP stdio connection named name.
> > LOCAL-COMMAND is either list of strings, string or function which
> returns the
> > command to execute."
> >   ;; Force a direct asynchronous process.
> >   (add-to-list 'tramp-connection-properties
> >        (list (regexp-quote (file-remote-p default-directory))
> >      "direct-async-process" t))
> >   (list :connect (lambda (filter sentinel name environment-fn)
> >                    (let* ((final-command (lsp-resolve-final-function
> local-command))
> >   (_stderr (or (when generate-error-file-fn
> >                                         (funcall generate-error-file-fn
> name))
> >                                       (format "/tmp/%s-%s-stderr" name
> >                                               (cl-incf
> lsp--stderr-index))))
> >                           (process-name (generate-new-buffer-name name))
> >                           (process-environment
> >                            (lsp--compute-process-environment
> environment-fn))
> >   (proc (make-process
> >  :name process-name
> >  :buffer (format "*%s*" process-name)
> >  :command final-command
> >  :connection-type 'pipe
> >  :coding 'no-conversion
> >  :noquery t
> >  :filter filter
> >  :sentinel sentinel
> >  :file-handler t)))
> >                      (cons proc proc)))
> >         :test? (lambda () (-> local-command lsp-resolve-final-function
> lsp-server-
> > present?)))) --8<---------------cut
> here---------------end--------------->8---
> >
> > With this, your test scenario works fine for me. Could you check how it
> fits in
> > your environment?
> >
> > One disadvantage is, that direct asynch processes work only with the
> > upcoming Tramp 2.5 (that's already in Emacs master), and it works only
> if the
> > asynchronous process does not require password handling. The latter is
> true,
> > if your ssh authentication is based on keys, or if you use Tramp's
> control
> > master arguments (enabled by default).
> >
> > > Hope this helps,
> > >
> > > Martin
> >
> > Best regards, Michael.
> This email and any attachments are intended solely for the use of the
> individual or entity to whom it is addressed and may be confidential and/or
> privileged.
>
> If you are not one of the named recipients or have received this email in
> error,
>
> (i) you should not read, disclose, or copy it,
>
> (ii) please notify sender of your receipt by reply email and delete this
> email and all attachments,
>
> (iii) Dassault Systèmes does not accept or assume any liability or
> responsibility for any use of or reliance on this email.
>
>
> Please be informed that your personal data are processed according to our
> data privacy policy as described on our website. Should you have any
> questions related to personal data protection, please contact 3DS Data
> Protection Officer at 3DS.compliance-privacy@3ds.com<mailto:
> 3DS.compliance-privacy@3ds.com>
>
>
> For other languages, go to https://www.3ds.com/terms/email-disclaimer
>

[-- Attachment #2: Type: text/html, Size: 14988 bytes --]

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

yyoncho <yyoncho@gmail.com> writes:

> Michael,

Hi Ivan,

> About the original issue: is it possible to avoid the issue using
> emacs 27.1(or older)?

Don't know. First, I need to make it work stable for Emacs 28.

> Asking the users to move to master branch is not
> very desirable since lsp-mode is used also by a lot of complete
> beginners.

Understood. I will check, but I cannot promise anything. I still don't
know, why the Content-Length line disappears with older Tramp. It is not
(only) an eol conversion problem, I believe.

> Thanks,
> Ivan

Best regards, Michael.

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Ivan,

>> About the original issue: is it possible to avoid the issue using
>> emacs 27.1(or older)?
>
> Don't know. First, I need to make it work stable for Emacs 28.
>
>> Asking the users to move to master branch is not
>> very desirable since lsp-mode is used also by a lot of complete
>> beginners.
>
> Understood. I will check, but I cannot promise anything. I still don't
> know, why the Content-Length line disappears with older Tramp. It is not
> (only) an eol conversion problem, I believe.

I made some tests, and it looks like Emacs 27.1 + Tramp 2.5 do work also
for lsp-mode. The latter one is the Tramp version integrated in Emacs
28, but I plan to release it via GNU ELPA end of the year.

>> Thanks,
>> Ivan

Best regards, Michael.

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

APEL Martin <Martin.APEL@3ds.com> writes:

> Hi Michael,

Hi Martin,

> I have found two places, where tramp-handle-make-process appears:
>
> 15:48:13.218370 tramp-handle-make-process (6) # ssh -q -l mal1 -e none
> dell248cem cd /scratch/apel/new_arch/ && env GIT_EDITOR=sh -c 'printf
> "WITH-EDITOR: $$ OPEN $0\037 IN $(pwd)\n"; sleep 604800 & sleep=$!;
> trap "kill $sleep; exit 0" USR1; trap "kill $sleep; exit 1" USR2; wait
> $sleep' git --no-pager --literal-pathspecs -c core.preloadindex=true
> -c log.showSignature=false -c color.ui=false -c color.diff=false
> commit --
> 15:57:44.334489 tramp-handle-make-process (6) # ssh -q -l mal1 -e none
> dell248cem cd /scratch/apel/new_arch/ && git --no-pager
> --literal-pathspecs -c core.preloadindex=true -c
> log.showSignature=false -c color.ui=false -c color.diff=false push -v
> origin branchname:refs/heads/branchname
>
> I have replaced the actual branchname by 'branchname' above.
>
> This is my last act at work for this year, so don't expect any reply soon ;-)

I've applied a patch. As far as I can tell, the magit problem has been
fixed (remember, I'm not a magit user).

> Best Regards,
>
> Martin

Best regards, Michael.

RE: Tramp and conversion of \r\n into \n

[APEL Martin]

Hi Michael,

thanks for the fix and happy new year to you. I have just pulled the latest changes and rebuilt. I can confirm, that the magit problem has disappeared and everything works as intended.
Thanks a lot for your help.

Best Regards,

Martin

> -----Original Message-----
> From: Michael Albinus <michael.albinus@gmx.de>
> Sent: 22 December 2020 15:58
> To: APEL Martin <Martin.APEL@3ds.com>
> Cc: emacs-devel@gnu.org
> Subject: Re: Tramp and conversion of \r\n into \n
>
> EXTERNAL EMAIL : The sender of this email is external to 3DS. Be wary of the
> content and do not open unexpected attachments or links. If you consider
> this email as spam, you can click the following link https://spam-
> report.3ds.com/?linktext=https://www.mailcontrol.com/sr/aIx62CWfcUDGX
> 2PQPOmvUnWvS4wCa9RdWH9lWLScXqCdpWQSps7DXI4GGZ5ZovpL89paDe1
> ZzCGrbIY9K1KtKg==  (no login or additional action will be requested).
>
>
> APEL Martin <Martin.APEL@3ds.com> writes:
>
> > Hi Michael,
>
> Hi Martin,
>
> > I have found two places, where tramp-handle-make-process appears:
> >
> > 15:48:13.218370 tramp-handle-make-process (6) # ssh -q -l mal1 -e none
> > dell248cem cd /scratch/apel/new_arch/ && env GIT_EDITOR=sh -c 'printf
> > "WITH-EDITOR: $$ OPEN $0\037 IN $(pwd)\n"; sleep 604800 & sleep=$!;
> > trap "kill $sleep; exit 0" USR1; trap "kill $sleep; exit 1" USR2; wait
> > $sleep' git --no-pager --literal-pathspecs -c core.preloadindex=true
> > -c log.showSignature=false -c color.ui=false -c color.diff=false
> > commit --
> > 15:57:44.334489 tramp-handle-make-process (6) # ssh -q -l mal1 -e none
> > dell248cem cd /scratch/apel/new_arch/ && git --no-pager
> > --literal-pathspecs -c core.preloadindex=true -c
> > log.showSignature=false -c color.ui=false -c color.diff=false push -v
> > origin branchname:refs/heads/branchname
> >
> > I have replaced the actual branchname by 'branchname' above.
> >
> > This is my last act at work for this year, so don't expect any reply
> > soon ;-)
>
> I've applied a patch. As far as I can tell, the magit problem has been fixed
> (remember, I'm not a magit user).
>
> > Best Regards,
> >
> > Martin
>
> Best regards, Michael.
This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systèmes does not accept or assume any liability or responsibility for any use of or reliance on this email.


Please be informed that your personal data are processed according to our data privacy policy as described on our website. Should you have any questions related to personal data protection, please contact 3DS Data Protection Officer at 3DS.compliance-privacy@3ds.com<mailto:3DS.compliance-privacy@3ds.com>


For other languages, go to https://www.3ds.com/terms/email-disclaimer

Re: Tramp and conversion of \r\n into \n

[Thomas Ross]

Hi Michael,

> Understood. I will check, but I cannot promise anything. I still don't
> know, why the Content-Length line disappears with older Tramp. It is not
> (only) an eol conversion problem, I believe.

I looked into this a bit and it seems like this isn't a TRAMP-specific
issue.

Running this Elisp snippet (after evaling both these sexps, open the
"hexdump" buffer to see the output -- you may need to scroll up to
actually see it) shows you that the carriage return gets replaced with a
line feed by Emacs:

> (make-process :name "hexdump" :command '("hexdump" "-v" "-e" "/1 \"%02X\n\"") :buffer "hexdump")
> (process-send-string "hexdump" "abc\r\n")

on my system (Linux, Emacs 27.1):

> 61
> 62
> 63
> 0A
> 0A

Note that there's 61 ("a"), 62 ("b"), 62 ("c"), and two 0As (LF). The
first 0A should really be a 0D (CR). I believe this conversion is likely
done by the call to encode_coding_object inside send_process
(process.c), but I might be wrong. I would need to sit down and step
through the code with gdb to confirm.

In a previous email in this thread (2020-12-16), you wrote:

> That is, both the Content-Length header is missing as well as the final "}".

I don't think the Content-Length header is actually missing. I think
clangd doesn't print it (even with --log=verbose). If you use the
wrapper script that Martin provided, I think you will see it in the logs
for standard input. The actual issue is that instead of sending this to
the process (all literal newlines are for display only):

> Content-Length: 2134<CR><LF>
> <CR><LF>
> {<...JSON here...>}<LF>

Emacs replaces the CRs with LFs, which confuses clangd:

> Content-Length: 2134<LF><LF>
> <LF><LF>
> {<...JSON here...>}<LF>

I verified this by looking at the hexdump of /tmp/clangd-logs (which
contains the data lsp-mode passes to clangd over stdin using the wrapper
script Martin provided).

Let me know what you think. It would be awesome if we were able to track
this issue down and have this all work without the direct async
processes (since there are a bunch of limitations there).

Thanks,
Thomas.

Re: Tramp and conversion of \r\n into \n

[Eli Zaretskii]

> From: Thomas Ross <thomasross@thomasross.io>
> Date: Wed, 4 Aug 2021 21:46:02 -0400
> Cc: APEL Martin <Martin.APEL@3ds.com>,
>  "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> 
> > (make-process :name "hexdump" :command '("hexdump" "-v" "-e" "/1 \"%02X\n\"") :buffer "hexdump")
> > (process-send-string "hexdump" "abc\r\n")
> 
> on my system (Linux, Emacs 27.1):
> 
> > 61
> > 62
> > 63
> > 0A
> > 0A
> 
> Note that there's 61 ("a"), 62 ("b"), 62 ("c"), and two 0As (LF). The
> first 0A should really be a 0D (CR). I believe this conversion is likely
> done by the call to encode_coding_object inside send_process
> (process.c), but I might be wrong. I would need to sit down and step
> through the code with gdb to confirm.

Is this in "emacs -Q"?  If so, such conversion should not happen, on
any OS.  Emacs converts CR on input, not on output, and
process-send-string does the latter.

> > Content-Length: 2134<CR><LF>
> > <CR><LF>
> > {<...JSON here...>}<LF>
> 
> Emacs replaces the CRs with LFs, which confuses clangd:

Again, this conversion can only happen on input, but you seem to be
talking about output.

It is possible that some Tramp magic causes this bytestream to be
input into Emacs at some point, as some temporary measure; in that
case, you should look for the problem at that point, where Emacs reads
these bytes, and disable decoding there.

Re: Tramp and conversion of \r\n into \n

[Stefan Monnier]

>> (make-process :name "hexdump" :command '("hexdump" "-v" "-e" "/1 \"%02X\n\"") :buffer "hexdump")
>> (process-send-string "hexdump" "abc\r\n")
>
> on my system (Linux, Emacs 27.1):
>
>> 61
>> 62
>> 63
>> 0A
>> 0A

What you're seeing here is the effect of the use of a tty (i.e. the
conversion or \r to \n is done by the tty rather than by Emacs).

Try to add `:connection-type 'pipe` to see the difference (you may also
have to add a call to `process-send-eof` to flush the send buffer, IIRC).


        Stefan

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

Hi Stefan,

> Try to add `:connection-type 'pipe` to see the difference (you may also
> have to add a call to `process-send-eof` to flush the send buffer, IIRC).

Yes, that makes a difference.

Thomas, Martin: does it help to let-bind tramp-process-connection-type
to 'pipe in lsp-tramp-connection?

>         Stefan

Best regards, Michael.

Re: Tramp and conversion of \r\n into \n

[Stefan Monnier]

>> Try to add `:connection-type 'pipe` to see the difference (you may also
>> have to add a call to `process-send-eof` to flush the send buffer, IIRC).
> Yes, that makes a difference.

As a general rule, I strongly recommend using pipes except when you know
you need a tty.  ttys have all kinds of quirks when it comes to
transferring "raw data".


        Stefan

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> As a general rule, I strongly recommend using pipes except when you know
> you need a tty.  ttys have all kinds of quirks when it comes to
> transferring "raw data".

Well, in Tramp the situation is different. It uses the default value t
for process-connection-type, because processes might ask for their
'remote-tty' process property.

>         Stefan

Best regards, Michael.

Re: Tramp and conversion of \r\n into \n

[Stefan Monnier]

Michael Albinus [2021-08-06 09:13:28] wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> As a general rule, I strongly recommend using pipes except when you know
>> you need a tty.  ttys have all kinds of quirks when it comes to
>> transferring "raw data".
> Well, in Tramp the situation is different. It uses the default value t
> for process-connection-type, because processes might ask for their
> 'remote-tty' process property.

I'm not sure I understand: do you mean that a Tramp client using
`start-file-process` specifying a nil `process-connection-type` may
still end up creating a process that uses a tty?


        Stefan

Re: Tramp and conversion of \r\n into \n

[Thomas Ross]

> Try to add `:connection-type 'pipe` to see the difference (you may also
> have to add a call to `process-send-eof` to flush the send buffer, IIRC).
Yep, looks like you're right. `:connection-type 'pipe` fixes this on my
local machine. It does not, however, fix it over TRAMP (and let-binding
tramp-process-connection-type to 'pipe doesn't change that):

> (make-process :name "hexdump" :command '("hexdump" "-v" "-e" "/1 \"%02X\n\"") :buffer "hexdump" :connection-type 'pipe :file-handler t)
> (process-send-string "hexdump" "abc\r\n"
> 61
> 62
> 63
> 0A
> 0A
It's also interesting to note that `(process-send-eof "hexdump")` is
required to get output on my local machine, but it's not required over
TRAMP.

Independent of this apparent TRAMP bug, lsp-tramp-connection will likely
need to be modified to use connection-type 'pipe. Currently it uses
start-file-process-shell-command (which calls start-file-process, which
calls start-process, which calls make-process), and uses the default
connection-type.

Thanks,
Thomas.

On 2021-08-05 10:10 a.m., Stefan Monnier wrote:
>>> (make-process :name "hexdump" :command '("hexdump" "-v" "-e" "/1 \"%02X\n\"") :buffer "hexdump")
>>> (process-send-string "hexdump" "abc\r\n")
>>
>> on my system (Linux, Emacs 27.1):
>>
>>> 61
>>> 62
>>> 63
>>> 0A
>>> 0A
> 
> What you're seeing here is the effect of the use of a tty (i.e. the
> conversion or \r to \n is done by the tty rather than by Emacs).
> 
> Try to add `:connection-type 'pipe` to see the difference (you may also
> have to add a call to `process-send-eof` to flush the send buffer, IIRC).
> 
> 
>         Stefan
> 

Re: Tramp and conversion of \r\n into \n

[Thomas Ross]

> Is this in "emacs -Q"?  If so, such conversion should not happen, on
> any OS.  Emacs converts CR on input, not on output, and
> process-send-string does the latter.

Yes, it is in `emacs -Q` -- this was determined to be done by the TTY
though, not Emacs. See Stefan's emails for more details. Sorry for the
confusion.

Thanks,
Thomas.

On 2021-08-05 2:16 a.m., Eli Zaretskii wrote:
>> From: Thomas Ross <thomasross@thomasross.io>
>> Date: Wed, 4 Aug 2021 21:46:02 -0400
>> Cc: APEL Martin <Martin.APEL@3ds.com>,
>>  "emacs-devel@gnu.org" <emacs-devel@gnu.org>
>>
>>> (make-process :name "hexdump" :command '("hexdump" "-v" "-e" "/1 \"%02X\n\"") :buffer "hexdump")
>>> (process-send-string "hexdump" "abc\r\n")
>>
>> on my system (Linux, Emacs 27.1):
>>
>>> 61
>>> 62
>>> 63
>>> 0A
>>> 0A
>>
>> Note that there's 61 ("a"), 62 ("b"), 62 ("c"), and two 0As (LF). The
>> first 0A should really be a 0D (CR). I believe this conversion is likely
>> done by the call to encode_coding_object inside send_process
>> (process.c), but I might be wrong. I would need to sit down and step
>> through the code with gdb to confirm.
> 
> Is this in "emacs -Q"?  If so, such conversion should not happen, on
> any OS.  Emacs converts CR on input, not on output, and
> process-send-string does the latter.
> 
>>> Content-Length: 2134<CR><LF>
>>> <CR><LF>
>>> {<...JSON here...>}<LF>
>>
>> Emacs replaces the CRs with LFs, which confuses clangd:
> 
> Again, this conversion can only happen on input, but you seem to be
> talking about output.
> 
> It is possible that some Tramp magic causes this bytestream to be
> input into Emacs at some point, as some temporary measure; in that
> case, you should look for the problem at that point, where Emacs reads
> these bytes, and disable decoding there.
> 

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Well, in Tramp the situation is different. It uses the default value t
>> for process-connection-type, because processes might ask for their
>> 'remote-tty' process property.
>
> I'm not sure I understand: do you mean that a Tramp client using
> `start-file-process` specifying a nil `process-connection-type` may
> still end up creating a process that uses a tty?

In general, a remote shell is opened, and the command is executed
there. So we have a tty.

An exception are the "direct async processes", introduced in Tramp
2.5. They call the command directly, w/o a remote shell. But they have
their limitations.

>         Stefan

Best regards, Michael.

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

Thomas Ross <thomasross@thomasross.io> writes:

Hi Thomas,

> Independent of this apparent TRAMP bug, lsp-tramp-connection will likely
> need to be modified to use connection-type 'pipe. Currently it uses
> start-file-process-shell-command (which calls start-file-process, which
> calls start-process, which calls make-process), and uses the default
> connection-type.

FTR, start-process is not involved. But yes, it might help to rewrite
lsp-tramp-connection to use make-process directly. And not to call any
remote shell, as it happens with start-file-process-shell-command.

> Thanks,
> Thomas.

Best regards, Michael.

Re: Tramp and conversion of \r\n into \n

[Stefan Monnier]

Michael Albinus [2021-08-07 15:51:43] wrote:
> In general, a remote shell is opened, and the command is executed
> there. So we have a tty.
>
> An exception are the "direct async processes", introduced in Tramp
> 2.5. They call the command directly, w/o a remote shell. But they have
> their limitations.

Ah, right , that makes sense.
I guess if we wanted to more faithfully simulate a pipe, we'd have to
send the process input via something like a base64 decoder and get the
output via a base64 encoder: this way not only stdin/stdout is a pipe but
we can protect ourselves from the way ttys can mess with some bytes (as
seen in this thread) and also overcome some of the limits of some ttys
when sending "large" amounts of data at a time (IIRC on some systems
this can either freeze the tty or the tty may insert extra chars like
^A or some such; my memory says the macOS kernel was such a culprits
some years ago).

The cure way be worse than the disease, tho.


        Stefan

PTYs vs. pipes for subprocesses [was: Tramp and conversion of \r\n into \n]

[tomas]

[-- Attachment #1: Type: text/plain, Size: 846 bytes --]

Hi,

another thing which has bitten me going from PTYs to pipes
(specifically because I wanted to have stderr in a separate
channel - that part worked nicely), is that the callee "sees"
that, thinks "oh, I'm not being called interactively [1] and
disables line buffering.

Interactivity goes out of the window. Not always welcome :-)

I "solved" that by invoking the callee through stdbuf. I didn't
know this even existed before. And it's ugly [2] and won't work
always. I don't dare yet to use that in other than toy applications
(and, of course, to get yourself out of the occasional hard
spot).

Cheers

[1] probably the classical isatty(3) dance.
[2] it basically works by sneaking itself into LD_PRELOAD
   Not that I'm not thankful for someone providing that,
   mind you. But it's not the typical "for production"
   thing, I'd say.

 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I guess if we wanted to more faithfully simulate a pipe, we'd have to
> send the process input via something like a base64 decoder and get the
> output via a base64 encoder: this way not only stdin/stdout is a pipe but
> we can protect ourselves from the way ttys can mess with some bytes (as
> seen in this thread) and also overcome some of the limits of some ttys
> when sending "large" amounts of data at a time (IIRC on some systems
> this can either freeze the tty or the tty may insert extra chars like
> ^A or some such; my memory says the macOS kernel was such a culprits
> some years ago).
>
> The cure way be worse than the disease, tho.

Yes.

Maybe we could play with "stty raw", when
(tramp-)process-connection-type is nil or 'pipe.

>         Stefan

Best regards, Michael.

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Thomas,

>> Independent of this apparent TRAMP bug, lsp-tramp-connection will likely
>> need to be modified to use connection-type 'pipe. Currently it uses
>> start-file-process-shell-command (which calls start-file-process, which
>> calls start-process, which calls make-process), and uses the default
>> connection-type.
>
> FTR, start-process is not involved.

That's wrong. start-process is called (inside tramp-maybe-open-connection).

>> Thanks,
>> Thomas.

Best regards, Michael.

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

[-- Attachment #1: Type: text/plain, Size: 2619 bytes --]

>> I guess if we wanted to more faithfully simulate a pipe, we'd have to
>> send the process input via something like a base64 decoder and get the
>> output via a base64 encoder: this way not only stdin/stdout is a pipe but
>> we can protect ourselves from the way ttys can mess with some bytes (as
>> seen in this thread) and also overcome some of the limits of some ttys
>> when sending "large" amounts of data at a time (IIRC on some systems
>> this can either freeze the tty or the tty may insert extra chars like
>> ^A or some such; my memory says the macOS kernel was such a culprits
>> some years ago).
>>
>> The cure way be worse than the disease, tho.
>
> Yes.
>
> Maybe we could play with "stty raw", when
> (tramp-)process-connection-type is nil or 'pipe.

It looks like "stty -icrnl" suppresses the \r\n conversion in a shell. I
have adapted Tramp to send this command to a remote async process, when
connection type is nil or 'pipe. The Tramp documentation has a new
subsection now:

--8<---------------cut here---------------start------------->8---
6.5.7 Remote process connection type

Asynchronous processes differ in the way, whether they use a pseudo tty,
or not.  This is controlled by the variable ‘process-connection-type’,
which can be ‘t’ or ‘pty’ (use a pseudo tty), or ‘nil’ or ‘pipe’ (don’t
use it).  TRAMP is based on running shells on the remote host, which
require a pseudo tty.  Therefore, it declares the variable
‘tramp-process-connection-type’, which carries this information for
remote processes.  Per default, its value is ‘t’.  The name of the
remote pseudo tty is returned by the function ‘process-tty-name’.

   If a remote process, started by ‘start-file-process’, shouldn’t use a
pseudo tty, this is emulated by let-binding this variable to ‘nil’ or
‘pipe’.  There is still a pseudo tty for the started process, but some
terminal properties are changed, like suppressing translation of
carriage return characters into newline.

   The function ‘make-process’ allows an explicit setting by the
‘:connection-type’ keyword.  If this keyword is not used, the value of
‘tramp-process-connection-type’ is applied instead.
--8<---------------cut here---------------end--------------->8---

The Tramp test suite has been extended in order to test this, see
tramp-test30-make-process. Committed to Emacs master, and will also be
available with next Tramp 2.5.1.2 on GNU ELPA.

Thomas, could you test this, again? I guess the following patch is all
what's needed in lsp-mode:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1523 bytes --]

*** /home/albinus/.emacs.d/elpa/lsp-mode-20210802.619/lsp-mode.el.~1~	2021-08-05 17:13:45.133161987 +0200
--- /home/albinus/.emacs.d/elpa/lsp-mode-20210802.619/lsp-mode.el	2021-08-12 20:41:28.232443184 +0200
***************
*** 6956,6962 ****
                                                                   (format "/tmp/%s-%s-stderr" name
                                                                           (cl-incf lsp--stderr-index))))))))
                            (process-environment
!                            (lsp--compute-process-environment environment-fn)))
                       (let ((proc (start-file-process-shell-command process-name
                                                                     (format "*%s*" process-name)
                                                                     wrapped-command)))
--- 6956,6963 ----
                                                                   (format "/tmp/%s-%s-stderr" name
                                                                           (cl-incf lsp--stderr-index))))))))
                            (process-environment
!                            (lsp--compute-process-environment environment-fn))
! 			  (tramp-process-connection-type 'pipe))
                       (let ((proc (start-file-process-shell-command process-name
                                                                     (format "*%s*" process-name)
                                                                     wrapped-command)))

[-- Attachment #3: Type: text/plain, Size: 24 bytes --]


Best regards, Michael.

Re: Tramp and conversion of \r\n into \n

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

> Thomas, could you test this, again? I guess the following patch is all
> what's needed in lsp-mode:

Well, lsp-tramp-connection uses start-file-process-shell-command.
Perhaps the shell is in the way, I recommend to use start-file-process
or make-process instead, if it doesn't work with my proposed patch.

Best regards, Michael.