safe way to add contents to a file ?

safe way to add contents to a file ?

[Jean-Christophe Helary]

I'm struggling with what is the ideal "elispy" way of adding contents to a file at a given position.

What I want it:
- create some content
- put that content at a specific location in a file saved on disk
 (current use case: add an xml block to an RSS file)

I thought write-region would do the trick but there are 2 issues with it

1) if APPEND is a number, that's the position from which my contents will *overwrite* the rest of the file (I want to *insert* my contents)

2) there are weird things like the whole code where I call this function is actually copied to the target file buffer and I have no idea how to prevent that.

So, I thought of something a bit convoluted:

use a temporary buffer
insert the contents of the file there
find the point where I want to insert my contents
find the end of the file
copy that region to a different buffer
write my contents from the point I specified above
append the copied region to that
write all that to the original file

But I thought, it's not like such things don't happen all the time in emacs, so there must be a better workflow.

But the reference is incredibly cryptic regarding that and the Intro does not address any file i/o at all...



Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: safe way to add contents to a file ?

[Óscar Fuentes]

Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
writes:

> I'm struggling with what is the ideal "elispy" way of adding contents
> to a file at a given position.
>
> What I want it:
> - create some content
> - put that content at a specific location in a file saved on disk
>  (current use case: add an xml block to an RSS file)
>
> I thought write-region would do the trick but there are 2 issues with
> it
>
> 1) if APPEND is a number, that's the position from which my contents
> will *overwrite* the rest of the file (I want to *insert* my contents)
>
> 2) there are weird things like the whole code where I call this
> function is actually copied to the target file buffer and I have no
> idea how to prevent that.
>
> So, I thought of something a bit convoluted:
>
> use a temporary buffer
> insert the contents of the file there

find-file-noselect takes care of creating the buffer and reading the
contents of the file. Later you must get rid of the buffer.

> find the point where I want to insert my contents

use `insert' or `insert-buffer-substring'

> write all that to the original file
>
> But I thought, it's not like such things don't happen all the time in
> emacs, so there must be a better workflow.
>
> But the reference is incredibly cryptic regarding that and the Intro
> does not address any file i/o at all...

The sequence of operations above is actually quite simple (although not
very efficient if the target file is large and your new content is near
the end).

Re: safe way to add contents to a file ?

[Jean-Christophe Helary]

Thank you Oscar.

> On Dec 18, 2019, at 9:36, Óscar Fuentes <ofv@wanadoo.es> wrote:
> 
> Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
> writes:
> 
>> I'm struggling with what is the ideal "elispy" way of adding contents
>> to a file at a given position.
>> 
>> What I want it:
>> - create some content
>> - put that content at a specific location in a file saved on disk
>> (current use case: add an xml block to an RSS file)
>> 
>> I thought write-region would do the trick but there are 2 issues with
>> it
>> 
>> 1) if APPEND is a number, that's the position from which my contents
>> will *overwrite* the rest of the file (I want to *insert* my contents)
>> 
>> 2) there are weird things like the whole code where I call this
>> function is actually copied to the target file buffer and I have no
>> idea how to prevent that.
>> 
>> So, I thought of something a bit convoluted:
>> 
>> use a temporary buffer
>> insert the contents of the file there
> 
> find-file-noselect takes care of creating the buffer and reading the
> contents of the file. Later you must get rid of the buffer.
> 
>> find the point where I want to insert my contents
> 
> use `insert' or `insert-buffer-substring'
> 
>> write all that to the original file
>> 
>> But I thought, it's not like such things don't happen all the time in
>> emacs, so there must be a better workflow.
>> 
>> But the reference is incredibly cryptic regarding that and the Intro
>> does not address any file i/o at all...
> 
> The sequence of operations above is actually quite simple (although not
> very efficient if the target file is large and your new content is near
> the end).

The problem I have is that nothing in the reference or in the doc strings or anywhere else I looked gives a clear path to discovering how to do that seemingly frequent action.

Or maybe there is something I missed.

Can you give me an indication on how to discover the process you described ?

Also, I don't understand the reason why write-region behaves the way I described:

>> 2) there are weird things like the whole code where I call this
>> function is actually copied to the target file buffer and I have no
>> idea how to prevent that.

Do you have an idea ?


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: safe way to add contents to a file ?

[Jean-Christophe Helary]

Ok, here is my attempt at this:


(setq myText "<item>bla</item>")
(setq myMarker "<!-- place new items before this comment -->")
(setq myFile "/path/to/test.xml")

(defun myInsert (myText myMarker myFile)
  (save-current-buffer
    (set-buffer (find-file-noselect myFile))
    (goto-char (point-min))
    (goto-char (- (search-forward myMarker) (length myMarker)))
    (insert myText)
    (save-buffer)
    (kill-buffer)))

(myInsert myText myMarker myFile)


I've used find-file-noselect and insert, as suggested.

Are there things I can do to make that code more idiomatic ? Is there anything I am seemingly not understanding ?

Thank you in advance.

Jean-Christophe 


> On Dec 18, 2019, at 9:36, Óscar Fuentes <ofv@wanadoo.es> wrote:
> 
> Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
> writes:
> 
>> I'm struggling with what is the ideal "elispy" way of adding contents
>> to a file at a given position.
>> 
>> What I want it:
>> - create some content
>> - put that content at a specific location in a file saved on disk
>> (current use case: add an xml block to an RSS file)
>> 
>> I thought write-region would do the trick but there are 2 issues with
>> it
>> 
>> 1) if APPEND is a number, that's the position from which my contents
>> will *overwrite* the rest of the file (I want to *insert* my contents)
>> 
>> 2) there are weird things like the whole code where I call this
>> function is actually copied to the target file buffer and I have no
>> idea how to prevent that.
>> 
>> So, I thought of something a bit convoluted:
>> 
>> use a temporary buffer
>> insert the contents of the file there
> 
> find-file-noselect takes care of creating the buffer and reading the
> contents of the file. Later you must get rid of the buffer.
> 
>> find the point where I want to insert my contents
> 
> use `insert' or `insert-buffer-substring'
> 
>> write all that to the original file
>> 
>> But I thought, it's not like such things don't happen all the time in
>> emacs, so there must be a better workflow.
>> 
>> But the reference is incredibly cryptic regarding that and the Intro
>> does not address any file i/o at all...
> 
> The sequence of operations above is actually quite simple (although not
> very efficient if the target file is large and your new content is near
> the end).
> 
> 

Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: safe way to add contents to a file ?

[Stefan Monnier]

> (setq myText "<item>bla</item>")
> (setq myMarker "<!-- place new items before this comment -->")
> (setq myFile "/path/to/test.xml")

Being at top-level these aren't just setting the vars but defining them,
so should use `defvar` or `defconst`.

> (defun myInsert (myText myMarker myFile)
>   (save-current-buffer
>     (set-buffer (find-file-noselect myFile))

`with-current-buffer` does the same, but shorter ;-)

>     (goto-char (point-min))
>     (goto-char (- (search-forward myMarker) (length myMarker)))

If the search fails, this will signal a "low-level" error, and it's
often useful to replace it with some other behavior (e.g. an error
message which the user is more likely to understand, or some other
behavior), so it's more idiomatic to do something like:

    (goto-char (point-min))
    (if (not (search-forward myMarker nil t))
        (user-error "Can't find foo bar in your fine file")
      (goto-char (match-beginning 0))

> Are there things I can do to make that code more idiomatic?

Can't think of anything else.


        Stefan

Re: safe way to add contents to a file ?

[Jean-Christophe Helary]

> On Dec 21, 2019, at 1:00, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
>> (setq myText "<item>bla</item>")
>> (setq myMarker "<!-- place new items before this comment -->")
>> (setq myFile "/path/to/test.xml")
> 
> Being at top-level these aren't just setting the vars but defining them,
> so should use `defvar` or `defconst`.
> 
>> (defun myInsert (myText myMarker myFile)
>>  (save-current-buffer
>>    (set-buffer (find-file-noselect myFile))
> 
> `with-current-buffer` does the same, but shorter ;-)

How is it supposed to be shorter ?

(defun myInsert (myText myMarker myFile)
  (save-current-buffer
    (set-buffer (find-file-noselect myFile))
    (goto-char (point-min))
    (goto-char (- (search-forward myMarker) (length myMarker)))
    (insert myText)
    (indent-region (point-min) (point-max))
    (save-buffer)
    (kill-buffer)))

(defun myInsert2 (myText myMarker myFile)
  (with-current-buffer
      (set-buffer (find-file-noselect myFile))
    (goto-char (point-min))
    (goto-char (- (search-forward myMarker) (length myMarker)))
    (insert myText)
    (indent-region (point-min) (point-max))
    (save-buffer)
    (kill-buffer)))


Honestly, as I read the respective doc strings and reference parts again, I'm not even sure why I chose save-current-buffer here anymore, but *because* there is no explanatory document on how to do proper file i/o in the Elisp Reference it is hard to make informed decisions, or even progress without a lot of searching outside the reference.

>>    (goto-char (point-min))
>>    (goto-char (- (search-forward myMarker) (length myMarker)))
> 
> If the search fails, this will signal a "low-level" error, and it's
> often useful to replace it with some other behavior (e.g. an error
> message which the user is more likely to understand, or some other
> behavior), so it's more idiomatic to do something like:
> 
>    (goto-char (point-min))
>    (if (not (search-forward myMarker nil t))
>        (user-error "Can't find foo bar in your fine file")
>      (goto-char (match-beginning 0))

But here, the code would go on inserting the text in a position that's not correct, right ?

Isn't it possible to catch the error around the search ? The only language I know, AppleScript, has a

try
  body
on error
  do something
end try

block which seems equivalent to "condition-case" but it seems that I'd have to use a progn to have the equivalent:

(defun myInsert3 (myText myMarker myFile)
  (save-current-buffer
    (set-buffer (find-file-noselect myFile))
    (goto-char (point-min))
    (condition-case nil
	(progn
	  (goto-char (- (search-forward myMarker) (length myMarker)))
	  (insert myText)
	  (indent-region (point-min) (point-max))
	  (save-buffer))
      (error (format "%s was not found" myMarker)))
    (kill-buffer)))

And then the message is not displayed. I just get a "t" which I guess corresponds to the successful (kill-buffer)...

And if I put the kill-buffer inside the progn, then I'm left with an open buffer that's not relevant anymore...


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: safe way to add contents to a file ?

[Jean-Christophe Helary]

> On Dec 22, 2019, at 12:14, Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org> wrote:
> 
> 
> 
>> On Dec 21, 2019, at 1:00, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> 
>>> (setq myText "<item>bla</item>")
>>> (setq myMarker "<!-- place new items before this comment -->")
>>> (setq myFile "/path/to/test.xml")
>> 
>> Being at top-level these aren't just setting the vars but defining them,
>> so should use `defvar` or `defconst`.
>> 
>>> (defun myInsert (myText myMarker myFile)
>>> (save-current-buffer
>>>   (set-buffer (find-file-noselect myFile))
>> 
>> `with-current-buffer` does the same, but shorter ;-)
> 
> How is it supposed to be shorter ?
> 
> (defun myInsert (myText myMarker myFile)
>  (save-current-buffer
>    (set-buffer (find-file-noselect myFile))
>    (goto-char (point-min))
>    (goto-char (- (search-forward myMarker) (length myMarker)))
>    (insert myText)
>    (indent-region (point-min) (point-max))
>    (save-buffer)
>    (kill-buffer)))
> 
> (defun myInsert2 (myText myMarker myFile)
>  (with-current-buffer
>      (set-buffer (find-file-noselect myFile))
>    (goto-char (point-min))
>    (goto-char (- (search-forward myMarker) (length myMarker)))
>    (insert myText)
>    (indent-region (point-min) (point-max))
>    (save-buffer)
>    (kill-buffer)))
> 
> 
> Honestly, as I read the respective doc strings and reference parts again, I'm not even sure why I chose save-current-buffer here anymore, but *because* there is no explanatory document on how to do proper file i/o in the Elisp Reference it is hard to make informed decisions, or even progress without a lot of searching outside the reference.
> 
>>>   (goto-char (point-min))
>>>   (goto-char (- (search-forward myMarker) (length myMarker)))
>> 
>> If the search fails, this will signal a "low-level" error, and it's
>> often useful to replace it with some other behavior (e.g. an error
>> message which the user is more likely to understand, or some other
>> behavior), so it's more idiomatic to do something like:
>> 
>>   (goto-char (point-min))
>>   (if (not (search-forward myMarker nil t))
>>       (user-error "Can't find foo bar in your fine file")
>>     (goto-char (match-beginning 0))
> 
> But here, the code would go on inserting the text in a position that's not correct, right ?
> 
> Isn't it possible to catch the error around the search ? The only language I know, AppleScript, has a
> 
> try
>  body
> on error
>  do something
> end try
> 
> block which seems equivalent to "condition-case" but it seems that I'd have to use a progn to have the equivalent:
> 
> (defun myInsert3 (myText myMarker myFile)
>  (save-current-buffer
>    (set-buffer (find-file-noselect myFile))
>    (goto-char (point-min))
>    (condition-case nil
> 	(progn
> 	  (goto-char (- (search-forward myMarker) (length myMarker)))
> 	  (insert myText)
> 	  (indent-region (point-min) (point-max))
> 	  (save-buffer))
>      (error (format "%s was not found" myMarker)))
>    (kill-buffer)))
> 
> And then the message is not displayed. I just get a "t" which I guess corresponds to the successful (kill-buffer)...
> 
> And if I put the kill-buffer inside the progn, then I'm left with an open buffer that's not relevant anymore...


Here is my latest version...


(defun myInsert4 (myText myMarker myFile)
  (save-current-buffer
    (set-buffer (find-file-noselect myFile))
    (goto-char (point-min))
    (if (not (search-forward myMarker nil t))
	(progn
	  (user-error (format "%s was not found" myMarker))
	  (kill-buffer))
      (progn
	(goto-char (point-min))
	(goto-char (- (search-forward myMarker) (length myMarker)))
	(insert myText)
	(indent-region (point-min) (point-max))
	(save-buffer)))
    (kill-buffer)))

Really not sure if I'm going in the right direction. Plus, for some reason, the buffer is not killed even after an error...


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: safe way to add contents to a file ?

[Óscar Fuentes]

Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
writes:

> Here is my latest version...
>
>
> (defun myInsert4 (myText myMarker myFile)
>   (save-current-buffer
>     (set-buffer (find-file-noselect myFile))
>     (goto-char (point-min))
>     (if (not (search-forward myMarker nil t))
> 	(progn
> 	  (user-error (format "%s was not found" myMarker))
> 	  (kill-buffer))
>       (progn
> 	(goto-char (point-min))
> 	(goto-char (- (search-forward myMarker) (length myMarker)))
> 	(insert myText)
> 	(indent-region (point-min) (point-max))
> 	(save-buffer)))
>     (kill-buffer)))
>
> Really not sure if I'm going in the right direction. Plus, for some
> reason, the buffer is not killed even after an error...

`user-error' (which is a variant of `error') stops the execution, so
`kill-buffer' is never executed. Either put `kill-buffer' before
`user-error' or do not use `user-error', like this:

(defun myInsert4 (myText myMarker myFile)
  (save-current-buffer (find-file-noselect myFile))
    (goto-char (point-min))
    (if (not (search-forward myMarker nil t))
        (message "%s was not found" myMarker)
      (progn
	(goto-char (point-min))
	(goto-char (- (search-forward myMarker) (length myMarker)))
	(insert myText)
	(indent-region (point-min) (point-max))
	(save-buffer)))
    (kill-buffer))

You can also cache the result of `search-forward', thus avoiding
repeating it:

(defun myInsert4 (myText myMarker myFile)
  (save-current-buffer (find-file-noselect myFile))
    (goto-char (point-min))
    (setq p (search-forward myMarker nil t))
    (if (not p)
        (message "%s was not found" myMarker)
      (progn
	(goto-char (- p (length myMarker)))
	(insert myText)
	(indent-region (point-min) (point-max))
	(save-buffer)))
    (kill-buffer))

A `let' would be nicer than a `setq'. Fixing that is left as an exercise
for the reader.

Re: safe way to add contents to a file ?

[Jean-Christophe Helary]

Oscar, Stephan,

Thank you *so* much for your comments.

Regarding `user-error' or `error', neither the doc strings nor the reference mention that the execution is stopped, I discovered that eventually but that was confusing. Am I missing something ?

Oscar, you say that a `let' would be "nicer" than a `setq', is it because `let' is local to the body that follows, so that I don't have to "polute" my top level with locally used values ?

Stephan, regarding `save-current-buffer' vs `with-file-noselect', the former saves the buffer that is current so I need to use `set-buffer' to set the current buffer, but `with-file-noselect' sets the current buffer to the buffer associated with the file, hence `set-buffer' is redundant there, right ?

`unwind-protect' is pretty cool ! I guess it can be used to easily handle errors without bothering about error messages. The doc strings are very clear, but the reference is totally confusing:

"The ‘unwind-protect’ construct is essential whenever you temporarily put
a data structure in an inconsistent state; it permits you to make the
data consistent again in the event of an error or throw.  (Another more
specific cleanup construct that is used only for changes in buffer
contents is the atomic change group; *note Atomic Changes::.)"

I would never think of using that form for such a trivial issue. Why is the writing so obfuscated ?

Jean-Christophe 



> On Dec 22, 2019, at 23:37, Óscar Fuentes <ofv@wanadoo.es> wrote:
> 
> Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
> writes:
> 
>> Here is my latest version...
>> 
>> 
>> (defun myInsert4 (myText myMarker myFile)
>>  (save-current-buffer
>>    (set-buffer (find-file-noselect myFile))
>>    (goto-char (point-min))
>>    (if (not (search-forward myMarker nil t))
>> 	(progn
>> 	  (user-error (format "%s was not found" myMarker))
>> 	  (kill-buffer))
>>      (progn
>> 	(goto-char (point-min))
>> 	(goto-char (- (search-forward myMarker) (length myMarker)))
>> 	(insert myText)
>> 	(indent-region (point-min) (point-max))
>> 	(save-buffer)))
>>    (kill-buffer)))
>> 
>> Really not sure if I'm going in the right direction. Plus, for some
>> reason, the buffer is not killed even after an error...
> 
> `user-error' (which is a variant of `error') stops the execution, so
> `kill-buffer' is never executed. Either put `kill-buffer' before
> `user-error' or do not use `user-error', like this:
> 
> (defun myInsert4 (myText myMarker myFile)
>  (save-current-buffer (find-file-noselect myFile))
>    (goto-char (point-min))
>    (if (not (search-forward myMarker nil t))
>        (message "%s was not found" myMarker)
>      (progn
> 	(goto-char (point-min))
> 	(goto-char (- (search-forward myMarker) (length myMarker)))
> 	(insert myText)
> 	(indent-region (point-min) (point-max))
> 	(save-buffer)))
>    (kill-buffer))
> 
> You can also cache the result of `search-forward', thus avoiding
> repeating it:
> 
> (defun myInsert4 (myText myMarker myFile)
>  (save-current-buffer (find-file-noselect myFile))
>    (goto-char (point-min))
>    (setq p (search-forward myMarker nil t))
>    (if (not p)
>        (message "%s was not found" myMarker)
>      (progn
> 	(goto-char (- p (length myMarker)))
> 	(insert myText)
> 	(indent-region (point-min) (point-max))
> 	(save-buffer)))
>    (kill-buffer))
> 
> A `let' would be nicer than a `setq'. Fixing that is left as an exercise
> for the reader.

Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: safe way to add contents to a file ?

[Óscar Fuentes]

Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
writes:

> Oscar, Stephan,
>
> Thank you *so* much for your comments.
>
> Regarding `user-error' or `error', neither the doc strings nor the
> reference mention that the execution is stopped, I discovered that
> eventually but that was confusing. Am I missing something ?

Strictly speaking, execution is not stopped but control goes searching
for a matching signal. The net effect on your case is that Emacs comes
back into interactive mode, which implies that the execution of your
command stops.

> Oscar, you say that a `let' would be "nicer" than a `setq', is it
> because `let' is local to the body that follows, so that I don't have
> to "polute" my top level with locally used values ?

Yes.

> Stephan, regarding `save-current-buffer' vs `with-file-noselect', the

BTW, my example was wrong, I meant to use

(with-current-buffer (find-file-no-select ...

Stefan did it right.

> former saves the buffer that is current so I need to use `set-buffer'
> to set the current buffer, but `with-file-noselect' sets the current
> buffer to the buffer associated with the file, hence `set-buffer' is
> redundant there, right ?

Correct, although you made up the name `with-file-noselect' when you
intended to mention `with-current-buffer' above.

> `unwind-protect' is pretty cool ! I guess it can be used to easily
> handle errors without bothering about error messages. The doc strings
> are very clear, but the reference is totally confusing:
>
> "The ‘unwind-protect’ construct is essential whenever you temporarily put
> a data structure in an inconsistent state; it permits you to make the
> data consistent again in the event of an error or throw.  (Another more
> specific cleanup construct that is used only for changes in buffer
> contents is the atomic change group; *note Atomic Changes::.)"
>
> I would never think of using that form for such a trivial issue.

Error handling is always under-appreciated by beginners (and many
veterans, sadly). When creating Emacs commands for your own use,
comprehensive error handling is not crucial, but for important code is
essential and a very difficult topic.

> Why is the writing so obfuscated ?

Actually, the text is trying to be helpful to non-expert readers. That's
always hard to achieve.

Re: safe way to add contents to a file ?

[Stefan Monnier]

> How is it supposed to be shorter ?
[...]
> (defun myInsert2 (myText myMarker myFile)
>   (with-current-buffer
>       (set-buffer (find-file-noselect myFile))

This `set-buffer` is redundant:

    (defun myInsert2 (myText myMarker myFile)
      (with-current-buffer (find-file-noselect myFile)

>>    (goto-char (point-min))
>>    (if (not (search-forward myMarker nil t))
>>        (user-error "Can't find foo bar in your fine file")
>>      (goto-char (match-beginning 0))
>
> But here, the code would go on inserting the text in a position that's not correct, right ?

No (for 2 reasons: the insertion would only in the `else` part of the
`if` and the `user-error` immediately terminates the execution of this function).

> And if I put the kill-buffer inside the progn, then I'm left with an
>  open buffer that's not relevant anymore...

Cleanup code like this `kill-buffer` should be placed in an
`unwind-protect`, so it's executed both for normal exits and for
non-local exits:

    (with-current-buffer (find-file-noselect myFile)
      (unwind-protect
          (progn
            ... do the insertion and stuff)
        (kill-buffer)))

Tho I'd change that code to pass the buffer explicitly to `kill-buffer`,
just to make sure there's no risk of killing some unrelated buffer in
fringe circumstances:

    (with-current-buffer (find-file-noselect myFile)
      (let ((buf (current-buffer)))
        (unwind-protect
            (progn
              ... do the insertion and stuff)
          (kill-buffer buf))))


-- Stefan

Re: safe way to add contents to a file ?

[Stefan Monnier]

> 1) if APPEND is a number, that's the position from which my contents will
> *overwrite* the rest of the file (I want to *insert* my contents)

This is the way life works, it's not specific to Emacs: given the way
files are typically represented on disk, there is simply no efficient
way to *insert* into a file.

> 2) there are weird things like the whole code where I call this function is
> actually copied to the target file buffer and I have no idea how to
> prevent that.

I don't understand what you're describing, really, but from where
I stand it sounds like you just had a weird bug in your code which made
it do something you did not intend.

> But I thought, it's not like such things don't happen all the time in emacs,
> so there must be a better workflow.

The normal workflow in Emacs is:
- read the file into a buffer.
- modify the buffer as you please (here, insertion is available and
  reasonably efficient).
- save the buffer back into the file.

The case you describe seems to fit this workflow perfectly and it'll be
pretty close to optimal compared to what could theoretically be obtained
from the POSIX API (my guess is that in the very worst case you'll be
performing twice the minimum number of disk accesses).

> But the reference is incredibly cryptic regarding that and the Intro does
> not address any file i/o at all...

Which part of which reference?  What have you looked for?

Given the structure of your problem, you have to read the file in order
to find the precise place where you want to insert the new content and
once you've read the file into a buffer, the rest seems to follow
quite naturally.


        Stefan

Re: safe way to add contents to a file ?

[Jean-Christophe Helary]

> On Dec 18, 2019, at 12:59, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
>> 1) if APPEND is a number, that's the position from which my contents will
>> *overwrite* the rest of the file (I want to *insert* my contents)
> 
> This is the way life works, it's not specific to Emacs: given the way
> files are typically represented on disk, there is simply no efficient
> way to *insert* into a file.

Maybe a note about that in the function reference would be a good reminder that disks are not like "shelves" or other places where you put "files" and "folders" IRL.

>> 2) there are weird things like the whole code where I call this function is
>> actually copied to the target file buffer and I have no idea how to
>> prevent that.
> 
> I don't understand what you're describing, really, but from where
> I stand it sounds like you just had a weird bug in your code which made
> it do something you did not intend.

The code is a most simple one liner variation on the following:

(write-region "stuff" nil "path/to/my/file/test.txt" nil t nil t).

Sometimes the whole *line* above (contained in file "test.el") would be copied to the file "test.txt" and I have no idea why.

>> But I thought, it's not like such things don't happen all the time in emacs,
>> so there must be a better workflow.
> 
> The normal workflow in Emacs is:
> - read the file into a buffer.
> - modify the buffer as you please (here, insertion is available and
>  reasonably efficient).
> - save the buffer back into the file.
> 
> The case you describe seems to fit this workflow perfectly and it'll be
> pretty close to optimal compared to what could theoretically be obtained
> from the POSIX API (my guess is that in the very worst case you'll be
> performing twice the minimum number of disk accesses).

Good to know. Thank you.

>> But the reference is incredibly cryptic regarding that and the Intro does
>> not address any file i/o at all...
> 
> Which part of which reference?

The elisp reference. I'm not aware that there is any other reference material regarding elisp. Is there ?

> What have you looked for?

The files chapter, the buffers chapter. It's all descriptions of function after function and overall very confusing when you don't know the basics of what's necessary.

> Given the structure of your problem, you have to read the file in order
> to find the precise place where you want to insert the new content and
> once you've read the file into a buffer, the rest seems to follow
> quite naturally.

It would certainly follow naturally if I were an elisp native. But thank you for the general confirmation that I'm going in the right direction.


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: safe way to add contents to a file ?

[Óscar Fuentes]

Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
writes:

>> What have you looked for?
>
> The files chapter, the buffers chapter. It's all descriptions of
> function after function and overall very confusing when you don't know
> the basics of what's necessary.

IMHO the Elisp reference is not the place to teach how file systems
handle content. As Stefan noted the Emacs workflow is not all that
different from what you could do with most popular APIs (POSIX, WIN32,
etc).

I think that the info nodes "Buffer basics" and the linked "Visiting
files" give a good hint about how to solve your task, supposing that you
know that on most operative systems it is not possible to directly
insert content on a file and you need to fake the operation.

Re: safe way to add contents to a file ?

[Jean-Christophe Helary]

Thank you for your comments.

> On Dec 18, 2019, at 22:10, Óscar Fuentes <ofv@wanadoo.es> wrote:
> 
> Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
> writes:
> 
>>> What have you looked for?
>> 
>> The files chapter, the buffers chapter. It's all descriptions of
>> function after function and overall very confusing when you don't know
>> the basics of what's necessary.
> 
> IMHO the Elisp reference is not the place to teach how file systems
> handle content.

Yes but I'm not writing binary (or even C) code here. I'm using a high-level language that is supposed to hide the complexity of all that machinery and help me think at a higher level.

> As Stefan noted the Emacs workflow is not all that
> different from what you could do with most popular APIs (POSIX, WIN32,
> etc).

No issue about that.

> I think that the info nodes "Buffer basics" and the linked "Visiting
> files" give a good hint about how to solve your task, supposing that you
> know that on most operative systems it is not possible to directly
> insert content on a file and you need to fake the operation.

I was naïvely thinking that inserting contents in a file is a frequent enough task (think xml nodes for ex), and actually even more frequent than appending, that there would be a relatively standard workflow proposed in the reference.

Stephan proposed that we consider inserting some extra info in the reference. When I'm done sorting my thing, I'll work on that.


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: safe way to add contents to a file ?

[Stefan Monnier]

> The code is a most simple one liner variation on the following:
>
> (write-region "stuff" nil "path/to/my/file/test.txt" nil t nil t).
>
> Sometimes the whole *line* above (contained in file "test.el") would be
> copied to the file "test.txt" and I have no idea why.

Very curious.  If you can come up with a recipe to reproduce it
somewhat-reliably, then please `M-x report-emacs-bug`.

>> Which part of which reference?
> The elisp reference. I'm not aware that there is any other reference
> material regarding elisp. Is there ?

I thought maybe you had found some material on a web page somewhere, but
other than that and the Emacs Lisp manual, there are of course other
books and last but not least docstrings (which is the reference I use
most of the time).

>> What have you looked for?
> The files chapter, the buffers chapter. It's all descriptions of function
> after function and overall very confusing when you don't know the basics of
> what's necessary.

Hmm... we do assume familiarity with how files work, yes.
Could propose a concrete change which would have helped you?


        Stefan

Re: safe way to add contents to a file ?

[Jean-Christophe Helary]

> On Dec 18, 2019, at 23:10, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
>>> What have you looked for?
>> The files chapter, the buffers chapter. It's all descriptions of function
>> after function and overall very confusing when you don't know the basics of
>> what's necessary.
> 
> Hmm... we do assume familiarity with how files work, yes.
> Could propose a concrete change which would have helped you?

I'll think about a proposal that when I'll have sorted what I did not understand. Thank you.

Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: safe way to add contents to a file ?

[Jean-Christophe Helary]

> On Dec 18, 2019, at 23:10, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
>> Sometimes the whole *line* above (contained in file "test.el") would be
>> copied to the file "test.txt" and I have no idea why.
> 
> Very curious.  If you can come up with a recipe to reproduce it
> somewhat-reliably, then please `M-x report-emacs-bug`.

It first happened in a piece of longer code I had written, then I wanted to isolate the file insertion part and improve/fix it so I just started working on that one line and the thing happened again.

I'll definitely report a bug when I can reproduce that because it is super weird. Thank you.


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune