From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Sean Devlin Newsgroups: gmane.emacs.bugs Subject: bug#69085: 29.2; Tramp: Extend tramp-make-copy-program-file-name via tramp-methods Date: Wed, 14 Feb 2024 19:57:13 -0500 Message-ID: <70162BD1-DC02-4838-9197-37249010B192@toadstyle.org> References: <327CF2FD-F00D-402B-B273-12066DDEB0E5@toadstyle.org> <875xytjsfw.fsf@gmx.de> <76B2D222-939F-4096-AFA5-2B0AA92A9A03@toadstyle.org> <8734twimm3.fsf@gmx.de> <87r0hfh6p1.fsf@gmx.de> Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.500.141\)) Content-Type: multipart/alternative; boundary="Apple-Mail=_53BEAA79-08C6-40E9-AB1A-16F98E517D08" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="31565"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 69085@debbugs.gnu.org To: Michael Albinus Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Thu Feb 15 01:59:01 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1raQ5d-000843-GI for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 15 Feb 2024 01:59:01 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1raQ5P-00051p-M7; Wed, 14 Feb 2024 19:58:48 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1raQ5M-00051f-Rh for bug-gnu-emacs@gnu.org; Wed, 14 Feb 2024 19:58:45 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1raQ5M-0002Uk-Iz for bug-gnu-emacs@gnu.org; Wed, 14 Feb 2024 19:58:44 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1raQ5e-0001z5-IN for bug-gnu-emacs@gnu.org; Wed, 14 Feb 2024 19:59:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Sean Devlin Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 15 Feb 2024 00:59:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 69085 X-GNU-PR-Package: emacs Original-Received: via spool by 69085-submit@debbugs.gnu.org id=B69085.17079587327601 (code B ref 69085); Thu, 15 Feb 2024 00:59:02 +0000 Original-Received: (at 69085) by debbugs.gnu.org; 15 Feb 2024 00:58:52 +0000 Original-Received: from localhost ([127.0.0.1]:53513 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1raQ5T-0001yW-QD for submit@debbugs.gnu.org; Wed, 14 Feb 2024 19:58:52 -0500 Original-Received: from mail-qk1-f169.google.com ([209.85.222.169]:51433) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1raQ5Q-0001yE-Vy for 69085@debbugs.gnu.org; Wed, 14 Feb 2024 19:58:50 -0500 Original-Received: by mail-qk1-f169.google.com with SMTP id af79cd13be357-7872bc61fcbso21922585a.2 for <69085@debbugs.gnu.org>; Wed, 14 Feb 2024 16:58:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toadstyle-org.20230601.gappssmtp.com; s=20230601; t=1707958645; x=1708563445; darn=debbugs.gnu.org; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:from:to:cc:subject:date:message-id:reply-to; bh=rpKDHDbZRs0YzmpjiZ3PVsXL1nYnu0iXD8II1CHQaN0=; b=eGsWDSDyUCclrLIKrJJGc3eNysC6JgdrauRKFAKbc5t963k6tnnY/2fo+juoDc4LRd gpr/mzDolwDekXAGc++g41ezzZc6ut6ArYWX3eQh14njocxVIVgOKe+hin+kkKyBrGtd RBQ3VRlxnQo9/qMr9lCkpbRMLQ6OICBbxZ8htuFdPDl+JegPqWDx9tfLycCaqGgA+MfH 4O5JK+OQwhx6lHF5pUsyKsRyFOOlviOIaT5OT4zgI0m1k6IK7ab3Jxo+UTNkJDjfTGm+ mSBNLVP+KvIFXE3AmNrmaBWNHqquLGZOaNXAIacDuYzWkL054/HaIlvafjbsfTySRC61 7D4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707958645; x=1708563445; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rpKDHDbZRs0YzmpjiZ3PVsXL1nYnu0iXD8II1CHQaN0=; b=AnmriGTET9RnbTYaTgvZR4bk0kScLDAI43UFUjY06InpXfEVNW1wYNaEApjuhOwTq8 uw62R5D1jnwlqiFOJIRBbloyGVe530zJZiHiIfvHyL3tDE0lfxAnO9hJhyRvmiCDhOj8 MNllDrv3EvTZrdQhzgQARV5cZzr15p/3P5ZDuw5sfVDZfm4mByWz/WMxLzvZ3/OCRH/j vM/O1ueLKGaqvI00o4U0cezC0j86K8VL9hhkuuHyQ3nHUfXcZow0oVm7/Jhr3wiGwLmS 0+pLtiiS0pufPTtYehKQIJWdt/zUfKT1prajVvgpnhXdNNd5X9PUVZzDCda/5JbIBFvx fwfQ== X-Gm-Message-State: AOJu0YxS9eznJwcohwSVv/rBQnUD7QVqyeQ4g+2JUbulFaWU8LP8eXk5 9ul6LpfdYkBzByfPVgQ9E++zKVr7XtZBegAV621fKfS5fin4dNi2tkoNAx3M0cdxNpxYX+ijEU2 v X-Google-Smtp-Source: AGHT+IHPLbuUbAeuU8x4LKFj0RoGVB7Q5yP9CeEifb/1riT37sBMEVg4Y96qN29lcZNlzpa2CS6vsg== X-Received: by 2002:a05:620a:2014:b0:785:d36b:a324 with SMTP id c20-20020a05620a201400b00785d36ba324mr397284qka.39.1707958645020; Wed, 14 Feb 2024 16:57:25 -0800 (PST) Original-Received: from smtpclient.apple ([70.107.172.48]) by smtp.gmail.com with ESMTPSA id po26-20020a05620a385a00b00785d4d5768bsm146386qkn.11.2024.02.14.16.57.24 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 14 Feb 2024 16:57:24 -0800 (PST) In-Reply-To: <87r0hfh6p1.fsf@gmx.de> X-Mailer: Apple Mail (2.3774.500.141) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:280033 Archived-At: --Apple-Mail=_53BEAA79-08C6-40E9-AB1A-16F98E517D08 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi Michael, > On Feb 14, 2024, at 1:59=E2=80=AFAM, Michael Albinus = wrote: >=20 > Sean Devlin writes: >=20 >> Hi Michael, >=20 > Hi Sean, >=20 >>> - And if HOST is a kind of container, would it be possible for you = to >>> use the "docker" or "podman" method of Tramp directly? >>=20 >> I=E2=80=99ve looked a little bit at the code supporting these methods = in tramp-container.el. >> They seem similar, but not exactly the same. Also, they do not = support OOB >> copying, at least in the version of Tramp I=E2=80=99m reading (i.e. = the version packaged with >> Emacs 29.2). >=20 > Could you pls elaborate verbosely, what is missing in your use case? > Perhaps we could extend tramp-container.el that it is applicable for = you > as well. Sure, let=E2=80=99s focus on tramp-container.el. I think this will allow = us to be concrete, which will make things simpler to discuss. Suppose we want to add OOB copying to the Docker method. Docker has a copy command with usage like this: Usage: docker cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH|- docker cp [OPTIONS] SRC_PATH|- CONTAINER:DEST_PATH We could try to add support to the Docker method for OOB copying by starting Emacs -Q and evaluating these forms: (require 'tramp-container) (add-to-list 'tramp-methods `(,tramp-docker-method (tramp-login-program ,tramp-docker-program) (tramp-login-args (("exec") ("-it") ("-u" "%u") ("%h") ("%l"))) (tramp-direct-async (,tramp-default-remote-shell "-c")) (tramp-remote-shell ,tramp-default-remote-shell) (tramp-remote-shell-login ("-l")) (tramp-remote-shell-args ("-i" "-c")) (tramp-copy-program ,tramp-docker-program) (tramp-copy-args (("cp"))) )) Outside of Emacs, start a Docker container running some image (e.g. ubuntu:latest). An example command is: $ docker run -it ubuntu:latest bash -l Leave this running. Next, create a large (i.e. greater than tramp-copy-size-limit) temporary file for copying: $ head -c 20000 /dev/random > /tmp/bigfile Back in Emacs, evaluate these forms (with =E2=80=9Cmy_container=E2=80=9D = replaced with the name of your container): (copy-file "/tmp/bigfile" "/docker:my_container:/tmp/bigfile") (copy-file "/tmp/bigfile" "/docker:root@my_container:/tmp/bigfile2=E2=80=9D= ) The first copy will succeed, but the second will fail. This is because = the connection has an explicit user, so it will try to run a command like: docker cp /tmp/bigfile root@my_container:/tmp/bigfile2 It will fail because it will interpret =E2=80=9Croot@my_container=E2=80=9D= as the container=E2=80=99s name, and it will say that there is no container by that name. I think what is needed is some way for the tramp-docker method to = specify the format for remote paths in the OOB copy command, since tramp-make-copy-program-file-name will include the user if it is = specified in the connection. One method might be to add a key to tramp-methods that lets you specify the format similar to tramp-login-args and tramp-copy-args: (tramp-copy-remote-file-format (("%h" ":" "%f"))) Where =E2=80=9C%h=E2=80=9D and =E2=80=9C%f=E2=80=9D are the host and the = file=E2=80=99s local name, respectively. Like tramp-login-args, the format could be a list of lists, and sublists = could be ignored when the required properties are absent. For example, the format for scp-like copy programs might look like this: (tramp-copy-remote-file-format (("%u" "@") ("%h" ":" "%f"))) Where =E2=80=9C%u=E2=80=9D is the user. So the leading =E2=80=9Cuser@=E2=80= =9C could be dropped if the user is unspecified. Unlike tramp-login-args and tramp-copy-args, this would need to concatenate the strings together without spaces. The function tramp-make-copy-program-file-name could check for this method parameter and, if present, use it to format the result. For = example, the cond at the end could be changed like so: (let ((fmt (tramp-get-method-parameter vec = 'tramp-copy-remote-file-format))) (cond ((tramp-get-method-parameter vec 'tramp-remote-copy-program) localname) (fmt ;; Format the file name using fmt. ) ((tramp-string-empty-or-nil-p user) (format "%s:%s" host localname)) (t (format "%s@%s:%s" user host localname))) Instead of a list of lists of strings, another method would be to = specify a function that does the work. The function could take the vector as input = and return the formatted file name. The tramp-make-copy-program-file-name function could call that function (if specified) at the same place in = that cond form. This would be more flexible than the list of lists of strings, but = it also would be more complicated for the implementer and may be overkill. >=20 > (I take the time to invest it with you, because your urgent problem = seems > to be mitigated by your defadvice.) I appreciate your help=E2=80=94thank you for taking the time! >=20 >> Thanks for your help! >=20 > Best regards, Michael. --Apple-Mail=_53BEAA79-08C6-40E9-AB1A-16F98E517D08 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hi Michael,

On Feb 14, 2024, at 1:59=E2=80=AFAM, Michael Albinus = <michael.albinus@gmx.de> wrote:

Sean Devlin = <spd@toadstyle.org> writes:

Hi = Michael,

Hi Sean,

- And if HOST is a kind of = container, would it be possible for you to
use the "docker" or = "podman" method of Tramp directly?

I=E2=80=99ve = looked a little bit at the code supporting these methods in = tramp-container.el.
They seem similar, but not exactly the same. = Also, they do not support OOB
copying, at least in the version of = Tramp I=E2=80=99m reading (i.e. the version packaged with
Emacs = 29.2).

Could you pls elaborate verbosely, what is = missing in your use case?
Perhaps we could extend tramp-container.el = that it is applicable for you
as = well.

Sure, let=E2=80=99s = focus on tramp-container.el. I think this will allow us to = be
concrete, which will make things simpler to = discuss.

Suppose we want to add OOB copying to = the Docker method. Docker
has a copy command with usage like = this:

Usage: =  docker cp [OPTIONS] CONTAINER:SRC_PATH = DEST_PATH|-
docker cp = [OPTIONS] SRC_PATH|- = CONTAINER:DEST_PATH

We could try to add = support to the Docker method for OOB copying by
starting Emacs = -Q and evaluating these forms:

(require = 'tramp-container)
(add-to-list 'tramp-methods
  =           =  `(,tramp-docker-method
        =        (tramp-login-program = ,tramp-docker-program)
          =      (tramp-login-args (("exec")
    =                     =           ("-it")
    =                     =           ("-u" "%u")
    =                     =           ("%h")
=          ("%l")))
  =     (tramp-direct-async (,tramp-default-remote-shell = "-c"))
              =  (tramp-remote-shell ,tramp-default-remote-shell)
  =             =  (tramp-remote-shell-login ("-l"))
      =          (tramp-remote-shell-args ("-i" = "-c"))

      = (tramp-copy-program ,tramp-docker-program)
  =     (tramp-copy-args (("cp")))
  =     ))

Outside of Emacs, start = a Docker container running some image (e.g.
ubuntu:latest). An = example command is:

$ docker run -it = ubuntu:latest bash -l

Leave this = running.

Next, create a large (i.e. greater = than tramp-copy-size-limit) temporary
file for = copying:

$ head -c 20000 /dev/random > = /tmp/bigfile

Back in Emacs, evaluate these = forms (with =E2=80=9Cmy_container=E2=80=9D replaced with
the = name of your container):

(copy-file = "/tmp/bigfile" = "/docker:my_container:/tmp/bigfile")

(copy-file = "/tmp/bigfile" = "/docker:root@my_container:/tmp/bigfile2=E2=80=9D)

The first copy will succeed, but the second will fail. This is = because the
connection has an explicit user, so it will try to = run a command like:

docker cp /tmp/bigfile = root@my_container:/tmp/bigfile2

It will fail = because it will interpret =E2=80=9Croot@my_container=E2=80=9D as the = container=E2=80=99s
name, and it will say that there is no = container by that name.

I think what is needed = is some way for the tramp-docker method to specify
the format = for remote paths in the OOB copy command, = since
tramp-make-copy-program-file-name will include the user = if it is specified
in the = connection.

One method might be to add a key to = tramp-methods that lets you specify
the format similar to = tramp-login-args and = tramp-copy-args:

(tramp-copy-remote-file-format = (("%h" ":" "%f")))

Where =E2=80=9C%h=E2=80=9D = and =E2=80=9C%f=E2=80=9D are the host and the file=E2=80=99s local name, = respectively.

Like tramp-login-args, the format = could be a list of lists, and sublists could
be ignored when = the required properties are absent. For example, the
format = for scp-like copy programs might look like = this:

(tramp-copy-remote-file-format
 (("%u" "@")
  ("%h" ":" = "%f")))

Where =E2=80=9C%u=E2=80=9D is the = user. So the leading =E2=80=9Cuser@=E2=80=9C could be dropped if = the
user is unspecified.

Unlike = tramp-login-args and tramp-copy-args, this would need = to
concatenate the strings together without = spaces.

The function = tramp-make-copy-program-file-name could check for this
method = parameter and, if present, use it to format the result. For = example,
the cond at the end could be changed like = so:

(let ((fmt (tramp-get-method-parameter = vec 'tramp-copy-remote-file-format)))
  = (cond
   ((tramp-get-method-parameter vec = 'tramp-remote-copy-program)
    = localname)
   (fmt
    ;; Format = the file name using fmt.
    )
  =  ((tramp-string-empty-or-nil-p user) (format "%s:%s" host = localname))
   (t (format "%s@%s:%s" user host = localname)))

Instead of a list of lists = of strings, another method would be to specify a
function that = does the work. The function could take the vector as input = and
return the formatted file name. The = tramp-make-copy-program-file-name
function could call that = function (if specified) at the same place in that cond
form. = This would be more flexible than the list of lists of strings, but it = also
would be more complicated for the implementer and may be = overkill.


(I take = the time to invest it with you, because your urgent problem seems
to = be mitigated by your = defadvice.)

I appreciate = your help=E2=80=94thank you for taking the time!


Thanks for your = help!

Best regards, = Michael.

= --Apple-Mail=_53BEAA79-08C6-40E9-AB1A-16F98E517D08--