TRAMP corrupting files

TRAMP corrupting files

[Will Willis]

I'm using Tramp from my Mac to edit remote files. Twice in the last
few days I've experienced Tramp corrupting my files.

The first corruption happened mid way through a PHP file, looking like this:

...
 <option value="C"<?=$uresult['store_cluster']=="C"?"
selected":""?>>Cluster C</option>
			  <opv˷⌀ “䛆需戀܀㜀䜀阀

I think the characters might have been encoded above. What I see in
emacs is this

...
 <option value="C"<?=$uresult['store_cluster']=="C"?"
selected":""?>>Cluster C</option>
			  <opv\367^B#\340\240\220\223\306...

This corruption goes on for 35,202 characters (all on one line) then
the file ends.


The second time I had a file get corrupted, interestingly also in a
PHP file inside of a <select> option group, There were not funky
characters, just a few dozen lines of missing from my file.

It's really starting to scare me as I'm not sure I can trust this
method of remote file editing... My hosting company was able to assist
me once in recovering a backup... I'd have for this to happen again..

Is TRAMP known to do this sort of thing?

Are there Emacs alternatives to remote file editing (w/o corruption)?

Thanks.

Carbon Emacs  "22.2.1"
Tramp "2.0.57"

Re: TRAMP corrupting files

[Ian Eure]

On May 7, 2009, at 10:44 PM, Will Willis wrote:

> I'm using Tramp from my Mac to edit remote files. Twice in the last
> few days I've experienced Tramp corrupting my files.
>
> The first corruption happened mid way through a PHP file, looking  
> like this:
>
> ...
> <option value="C"<?=$uresult['store_cluster']=="C"?"
> selected":""?>>Cluster C</option>
> 			  <opv˷⌀ “䛆需戀܀㜀䜀阀
>
> I think the characters might have been encoded above. What I see in
> emacs is this
>
> ...
> <option value="C"<?=$uresult['store_cluster']=="C"?"
> selected":""?>>Cluster C</option>
> 			  <opv\367^B#\340\240\220\223\306...
>
> This corruption goes on for 35,202 characters (all on one line) then
> the file ends.
>
>
> The second time I had a file get corrupted, interestingly also in a
> PHP file inside of a <select> option group, There were not funky
> characters, just a few dozen lines of missing from my file.
>
> It's really starting to scare me as I'm not sure I can trust this
> method of remote file editing... My hosting company was able to assist
> me once in recovering a backup... I'd have for this to happen again..
>
> Is TRAMP known to do this sort of thing?
>
For what it's worth, I live and breathe Tramp. The vast majority of my  
editing happens remotely via tramp, and I have _never_ experienced  
anything like what you have. Which is not to say it can never happen,  
but I don't think it should in the normal course of events.

I'm currently using CVS snapshots of Carbon Emacs 23 on two Macs.  
Before that, I was using Aquamacs, and XEmacs 20/21 on Debian before  
that. The remote hosts are a mix of Lenny and Etch Debian boxes. I've  
used the rsync and ssh Tramp methods.

  - Ian

Re: TRAMP corrupting files

[Will Willis]

On Mon, May 18, 2009 at 5:35 PM, Ian Eure <ian@digg.com> wrote:
> On May 7, 2009, at 10:44 PM, Will Willis wrote:
>
>> I'm using Tramp from my Mac to edit remote files. Twice in the last
>> few days I've experienced Tramp corrupting my files.
>>
>> The first corruption happened mid way through a PHP file, looking like
>> this:
>>
>> ...
>> <option value="C"<?=$uresult['store_cluster']=="C"?"
>> selected":""?>>Cluster C</option>
>>                          <opv˷⌀   “䛆需戀܀㜀䜀阀
>>
>> I think the characters might have been encoded above. What I see in
>> emacs is this
>>
>> ...
>> <option value="C"<?=$uresult['store_cluster']=="C"?"
>> selected":""?>>Cluster C</option>
>>                          <opv\367^B#\340\240\220\223\306...
>>
>> This corruption goes on for 35,202 characters (all on one line) then
>> the file ends.
>>
>>
>> The second time I had a file get corrupted, interestingly also in a
>> PHP file inside of a <select> option group, There were not funky
>> characters, just a few dozen lines of missing from my file.
>>
>> It's really starting to scare me as I'm not sure I can trust this
>> method of remote file editing... My hosting company was able to assist
>> me once in recovering a backup... I'd have for this to happen again..
>>
>> Is TRAMP known to do this sort of thing?
>>
> For what it's worth, I live and breathe Tramp. The vast majority of my
> editing happens remotely via tramp, and I have _never_ experienced anything
> like what you have. Which is not to say it can never happen, but I don't
> think it should in the normal course of events.
>
> I'm currently using CVS snapshots of Carbon Emacs 23 on two Macs. Before
> that, I was using Aquamacs, and XEmacs 20/21 on Debian before that. The
> remote hosts are a mix of Lenny and Etch Debian boxes. I've used the rsync
> and ssh Tramp methods.
>
>  - Ian

I'm going to start using rsync to see if that helps.. I wish I were
making this up. I've had files corrupt on me as many as 3 times in one
night. I'd love to upgrade to Carbon Emacs 23, but I'm still on OS X
10.4, holding out for snow leopard.

-Will

Re: TRAMP corrupting files (it's OS X's fault)

[Daniel Colascione]

On May 18, 2009, at 9:40 PM, Will Willis wrote:
> I'm going to start using rsync to see if that helps.. I wish I were
> making this up. I've had files corrupt on me as many as 3 times in one

I ran into the same problem. It's actually OS X's fault: OS X (at  
least 10.5) doesn't give Emacs any locale environment variables. Emacs  
guesses it's running under 8859-1, which has disastrous consequences  
for tramp. (If you want to test it, run locale under eshell. In the  
terminal, you see the expected locale output because Terminal.app  
*itself* sets up the correct environment before running your shell.)

One solution is to just give Emacs the right environment variables.  
I've outlined how to do that here:

http://qtmstr.livejournal.com/19155.html

(Modify the specific environment variable values according to taste.)

Another approach is to add this to your Emacs initialization:

(when (eq system-type 'darwin)
     (prefer-coding-system 'utf-8))

These fixed the corruption issue *I* was having. YMMV.

> I'd love to upgrade to Carbon Emacs 23, but I'm still on OS X
> 10.4, holding out for snow leopard.

There is no CarbonEmacs 23. The OS X support will be provided via a  
new Cocoa port which, to put it kindly, is broken. You might want to  
stick with CarbonEmacs for now.

Re: TRAMP corrupting files (it's OS X's fault)

[Will Willis]

Thank you so much for the suggestion!!

-Will

On Tue, May 19, 2009 at 12:17 AM, Daniel Colascione
<danc@merrillpress.com> wrote:
> On May 18, 2009, at 9:40 PM, Will Willis wrote:
>>
>> I'm going to start using rsync to see if that helps.. I wish I were
>> making this up. I've had files corrupt on me as many as 3 times in one
>
> I ran into the same problem. It's actually OS X's fault: OS X (at least
> 10.5) doesn't give Emacs any locale environment variables. Emacs guesses
> it's running under 8859-1, which has disastrous consequences for tramp. (If
> you want to test it, run locale under eshell. In the terminal, you see the
> expected locale output because Terminal.app *itself* sets up the correct
> environment before running your shell.)
>
> One solution is to just give Emacs the right environment variables. I've
> outlined how to do that here:
>
> http://qtmstr.livejournal.com/19155.html
>
> (Modify the specific environment variable values according to taste.)
>
> Another approach is to add this to your Emacs initialization:
>
> (when (eq system-type 'darwin)
>    (prefer-coding-system 'utf-8))
>
> These fixed the corruption issue *I* was having. YMMV.
>
>> I'd love to upgrade to Carbon Emacs 23, but I'm still on OS X
>> 10.4, holding out for snow leopard.
>
> There is no CarbonEmacs 23. The OS X support will be provided via a new
> Cocoa port which, to put it kindly, is broken. You might want to stick with
> CarbonEmacs for now.
>

Re: TRAMP corrupting files (it's OS X's fault)

[Michael Albinus]

Daniel Colascione <danc@merrillpress.com> writes:

> On May 18, 2009, at 9:40 PM, Will Willis wrote:
>> I'm going to start using rsync to see if that helps.. I wish I were
>> making this up. I've had files corrupt on me as many as 3 times in one
>
> I ran into the same problem. It's actually OS X's fault: OS X (at
> least 10.5) doesn't give Emacs any locale environment variables. Emacs
> guesses it's running under 8859-1, which has disastrous consequences
> for tramp. (If you want to test it, run locale under eshell. In the
> terminal, you see the expected locale output because Terminal.app
> *itself* sets up the correct environment before running your shell.)

Tramp cannot solve the problem, as said. But it might be worth to know,
that Tramp supports the `file-precious-flag' variable. If it is set to
`t', Tramp signals an error, when the remote file is corrupted during
copying.

Best regards, Michael.

Re: TRAMP corrupting files (it's OS X's fault)

[Daniel Colascione]

On Tuesday 26 May 2009, Michael Albinus wrote:
> Tramp cannot solve the problem, as said. But it might be worth to know,
> that Tramp supports the `file-precious-flag' variable. If it is set to
> `t', Tramp signals an error, when the remote file is corrupted during
> copying.

Why does file-precious-flag even exist? Isn't the behavior you get when it's t the behavior you *always* want when the combination of the various backup-by-* variables allows it?

Re: TRAMP corrupting files (it's OS X's fault)

[Michael Albinus]

Daniel Colascione <danc@merrillpress.com> writes:

> On Tuesday 26 May 2009, Michael Albinus wrote:
>> Tramp cannot solve the problem, as said. But it might be worth to know,
>> that Tramp supports the `file-precious-flag' variable. If it is set to
>> `t', Tramp signals an error, when the remote file is corrupted during
>> copying.
>
> Why does file-precious-flag even exist? Isn't the behavior you get
> when it's t the behavior you *always* want when the combination of the
> various backup-by-* variables allows it?

Yes, you can have the same effect with backup-by-* variables. If you
read the code of backup-buffer, you see

      (if (or file-precious-flag
	      backup-by-copying
	      (and backup-by-copying-when-linked
		   (> (file-nlinks real-file-name) 1))
	      (and (or backup-by-copying-when-mismatch
		       (integerp backup-by-copying-when-privileged-mismatch))
              ...

I believe, there are historical reasons for the existence of that
variable. Tramp just has occupied the variable because it exists,
avoiding introduction of another customer option.

Best regards, Michael.