Re: TODO crdt-stop-session, selecting deleted buffer, fails to remove session from the list

Re: TODO crdt-stop-session, selecting deleted buffer, fails to remove session from the list

[Jean Louis]

I think you should at least post this to emacs-tangents@gnu.org as it
is related to Emacs, not yet to Emacs development. I make copy there,
OK? There could be people with good ideas.

You have that account on Savannah to contribute to Emacs or ELPA? I
hope so.

* Qiantan Hong <qhong@mit.edu> [2020-10-22 02:31]:
> I find that it require some non trivial refactoring to do this
> — currently I just assume that client always open any buffer server is sharing.
> 
> So to solve the problem there’s two way:
> 
> - Keep the invariant "client always open any buffer server is sharing”.
> If client try to kill a shared buffer, warn them and say if doing so, it will
> cause the client to leave the whole session. This is very simple to do.
> 
> - Change the model and not keeping the invariant
> "client always open any buffer server is sharing”. This will require
> non-trivial work. After doing so, however, we can allow feature like
> lazily pull buffer from server (only when the client need it).
> 
> What do you think?

My suggestion is that you install Gobby collaborative editor and look
inside, I did, then follow the well established model, you almost have
it by same model.

My opinion is that client should connect:

- immediately to the buffer shared, if it is the only one

- to the selection of buffers if there are multiple

- and to be given option to connect to other available buffers, but
  not automatically

> - Keep the invariant "client always open any buffer server is sharing”.
> If client try to kill a shared buffer, warn them and say if doing so, it will
> cause the client to leave the whole session. This is very simple to do.

It is not good to kill the session by closing the buffer, as client
did not decide to kill the session, and in the mean time, server could
provide other files, right?

It is better to ask user if the session should be also disconnected,
and not do it automatically.

If there are other available files that appeared in the mean time,
user shall be informed in that message.

Generally option 2 seem logical more than option 1, which looks not
well planned for multi users having multi sessions with multi buffers.

Jean

crdt.el: proposal that server enforces the mode

[Jean Louis]

I do not know why, but often when I go do some personal hygiene, my
mind get new ideas. I am programming in Emacs Lisp, but not internals
of Emacs, so I cannot know if following is right or wrong.

I was thinking of various modes, likes ses-mode for spreadsheet or Org
mode, there is also artist mode and other kind of special modes.

It would not be good if client can choose the mode and change it, that
would influence data, it could be disaster.

So in that sense, the mode of editing should be set and dictated by
the server.

Now, what if client does not have some of the modes? This condition
also has to be handled, maybe there should be option

crdt-allow-client-with-non-existant-mode or similar

As maybe some modes are just fine to be edited on server let us say in
Org mode, and fundamental mode could be used on client's side.

From Internet searches and report of my friend, I got impression that
major collaboration is on spreadsheet, that is why there is Ethercalc
software, it works online. So if it is on spreadsheets, people wish to
insert their statistics or reports. Spreadsheets are exactly related
to this proposal.

ses-mode handles spreadsheets, there may be dismal spreadsheet, and
there may be table modes in Emacs.

For example, in Org mode, table handling, if I press TAB, I can all
the table nicely aligned, if I press TAB in mail-mode I get several
spaces, it would not be same thing, you will say more about that.

Re: crdt.el: proposal that server enforces the mode

[Qiantan Hong]

[-- Attachment #1: Type: text/plain, Size: 3131 bytes --]

I thought about exactly this, and this come up when I was
discussing read-only mode with my friends. Currently it doesn’t
interact well with crdt.el

Currently artist-mode works fine with crdt.el (and it’s fun!), but
if variables like tab-width have different value on different peers
it doesn’t behave well.

That leads me to the idea of synchronizing buffer local variables, and
more generally: (copied from my post on /r/emacs)

<start quote>

My real long-term objective is Emacs as an collaborative operating system — 
with a few annotations, developer should be able to make any Emacs application 
collaboration-powered. This is based on the observation that lots of Emacs application
is centered around the buffer and buffer-local-variables, and I imagine by implementing 
synchronization primitives for all components in a buffer pretty much everything 
can be made collaborative.

See the crdt-org-sync-overlay-mode as a prototypical example. It's powered by the
synchronization of overlays (also based on CRDT). IMO once the synchronization 
of buffer text, text-properties, overlays, markers and buffer-local-variables (with annotation) 
is all implemented, it could be much more powerful than any existing collaborative environment.

<end quote>

I’m not sure about the idea of letting the server “dictate” anything.
Emotionally I don’t like it — I try to avoid power hierarchy. And technically,
I can synchronize Boolean variable also using CRDT.

> On Oct 21, 2020, at 7:54 PM, Jean Louis <bugs@gnu.support> wrote:
> 
> I do not know why, but often when I go do some personal hygiene, my
> mind get new ideas. I am programming in Emacs Lisp, but not internals
> of Emacs, so I cannot know if following is right or wrong.
> 
> I was thinking of various modes, likes ses-mode for spreadsheet or Org
> mode, there is also artist mode and other kind of special modes.
> 
> It would not be good if client can choose the mode and change it, that
> would influence data, it could be disaster.
> 
> So in that sense, the mode of editing should be set and dictated by
> the server.
> 
> Now, what if client does not have some of the modes? This condition
> also has to be handled, maybe there should be option
> 
> crdt-allow-client-with-non-existant-mode or similar
> 
> As maybe some modes are just fine to be edited on server let us say in
> Org mode, and fundamental mode could be used on client's side.
> 
> From Internet searches and report of my friend, I got impression that
> major collaboration is on spreadsheet, that is why there is Ethercalc
> software, it works online. So if it is on spreadsheets, people wish to
> insert their statistics or reports. Spreadsheets are exactly related
> to this proposal.
> 
> ses-mode handles spreadsheets, there may be dismal spreadsheet, and
> there may be table modes in Emacs.
> 
> For example, in Org mode, table handling, if I press TAB, I can all
> the table nicely aligned, if I press TAB in mail-mode I get several
> spaces, it would not be same thing, you will say more about that.


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 1858 bytes --]

Re: crdt.el: proposal that server enforces the mode\

[Jean Louis]

* Qiantan Hong <qhong@mit.edu> [2020-10-22 03:05]:
> I thought about exactly this, and this come up when I was
> discussing read-only mode with my friends. Currently it doesn’t
> interact well with crdt.el
> 
> Currently artist-mode works fine with crdt.el (and it’s fun!), but
> if variables like tab-width have different value on different peers
> it doesn’t behave well.

You need to make set of variables that will be dictated by server and
buffer local set on each of clients. This list may grow with the time.

> See the crdt-org-sync-overlay-mode as a prototypical example. It's
> powered by the synchronization of overlays (also based on CRDT). IMO
> once the synchronization of buffer text, text-properties, overlays,
> markers and buffer-local-variables (with annotation) is all
> implemented, it could be much more powerful than any existing
> collaborative environment.

Exactly that is it.

> I’m not sure about the idea of letting the server “dictate”
> anything.  Emotionally I don’t like it — I try to avoid power
> hierarchy. And technically, I can synchronize Boolean variable also
> using CRDT.

dictate

* Overview of verb dictate

The verb dictate has 3 senses (first 2 from tagged texts)
1. (8) order, prescribe, dictate -- (issue commands or orders for)
2. (1) dictate -- (say out loud for the purpose of recording; "He dictated a report to his secretary")
3. dictate -- (rule as a dictator)

My usage of dictate is in the sense of definition 1, not 3, so when
you press a key in Emacs you already issue some commands, if you share
a buffer, you dictate that it is shared, right?

crdt.el shall be collaborative, with many factors in synchronization.

My proposal is directed for those using ses mode, table modes, org
mode, and maybe other dismal mode or other spreadsheet modes, as that
is what people use on proprietary software, like Microsoft Excell.

If that feature is implemented, then subset of those people could
start using free software.

Re: crdt.el: proposal that server enforces the mode\

[Qiantan Hong]

[-- Attachment #1: Type: text/plain, Size: 1452 bytes --]

> 
>> I’m not sure about the idea of letting the server “dictate”
>> anything.  Emotionally I don’t like it — I try to avoid power
>> hierarchy. And technically, I can synchronize Boolean variable also
>> using CRDT.
> 
> dictate
> 
> * Overview of verb dictate
> 
> The verb dictate has 3 senses (first 2 from tagged texts)
> 1. (8) order, prescribe, dictate -- (issue commands or orders for)
> 2. (1) dictate -- (say out loud for the purpose of recording; "He dictated a report to his secretary")
> 3. dictate -- (rule as a dictator)
> 
> My usage of dictate is in the sense of definition 1, not 3, so when
> you press a key in Emacs you already issue some commands, if you share
> a buffer, you dictate that it is shared, right?
> 
> crdt.el shall be collaborative, with many factors in synchronization.
> 
So in this occasion particularly, I’m asking whether should the server
decide exclusively what minor-mode to enable, or values of buffer local variables,
or any client can change it?

For Boolean variables (like on/off of minor modes), I can give write access to
every peer and synchronize it using PN counter CRDT. I can also do similar
thing to some other buffer local variables with the help of some annotations.

However, (although I don’t like it very much), I think one can argue that giving
server exclusive write permission avoid communicational conflicts or even chaos.
What do you think?

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 1858 bytes --]

Re: crdt.el: proposal that server enforces the mode\

[Jean Louis]

* Qiantan Hong <qhong@mit.edu> [2020-10-22 03:17]:
> So in this occasion particularly, I’m asking whether should the server
> decide exclusively what minor-mode to enable, or values of buffer local variables,
> or any client can change it?

You have to select what would be set by server only and that list of
variables should be later explained in the info file or README. It
should not be general, it could make anything insecure or weird if you
make it general.

In my opinion those variables dictated by server should be in the
customization group, so that user can remove the variable or set it.

If client can change it, it makes no much sense really, not to me, but
it may make sense to somebody.

If I use ses-mode, I need spreadsheet and not damages.

> However, (although I don’t like it very much), I think one can argue that giving
> server exclusive write permission avoid communicational conflicts or even chaos.
> What do you think?

Server need not be necessarily manager or leader of the team, but
leader will like to decide upon those variables to be set by the
server.

Thus my opinion is that customization group should be there where
server's user can set it up on what to be dictated by the server, for
example modes, variables, etc.

crdt.el: include chat buffer

[Jean Louis]

Proposal is that you make additional chat buffer, where lines by each
users are simply appended.

When buffers are destroyed or upon last buffer destroyed the chat
buffer should remain.

Chat buffer offers collaboration line beyong editing buffer and users
do need communication in collaboration.

Chat buffer should open by default, and it should be customizable to
turn it off in general, as some people may use video or sound
collaboration, which tools may already have their own text chat as
well.

Jean

Re: crdt.el: include chat buffer

[Aldric Giacomoni]

[-- Attachment #1: Type: text/plain, Size: 778 bytes --]

Is this truly the best place to have a conversation about a single emacs
package? Surely the project has a dedicated space for this.

--

Aldric.
Sent from a mobile device.

On Wed, Oct 21, 2020, 20:32 Jean Louis <bugs@gnu.support> wrote:

> Proposal is that you make additional chat buffer, where lines by each
> users are simply appended.
>
> When buffers are destroyed or upon last buffer destroyed the chat
> buffer should remain.
>
> Chat buffer offers collaboration line beyong editing buffer and users
> do need communication in collaboration.
>
> Chat buffer should open by default, and it should be customizable to
> turn it off in general, as some people may use video or sound
> collaboration, which tools may already have their own text chat as
> well.
>
> Jean
>
>

[-- Attachment #2: Type: text/html, Size: 1108 bytes --]

Re: crdt.el: include chat buffer

[Jean Louis]

* Aldric Giacomoni <trevoke@gmail.com> [2020-10-22 03:33]:
> Is this truly the best place to have a conversation about a single emacs
> package? Surely the project has a dedicated space for this.

Looks like it is, crdt.el is important.

See here:
http://savannah.gnu.org/mail/?group=emacs

Emacs news and miscellaneous discussions outside the scope of other
Emacs mailing lists



Jean

Re: TODO crdt-stop-session, selecting deleted buffer, fails to remove session from the list

[Jean Louis]

I will post this to emacs-tangents, maybe other people wish to
contribute to this package.

* Qiantan Hong <qhong@mit.edu> [2020-10-23 05:42]:
> Just implemented option 2 and pushed.
> 
> Also talking about savannah, I’ve just registered an account.
> Any further action to take?
> Also our school’s copyright office is being super slow these days
> and I haven’t got my disclaimer yet.

Alright, regarding that you please communicate to eliz AT gnu.org Eli
Zaretskii or somebody else.

I am not in GNU project neither represent GNU project, I am just
interested to have collaboration.

By the way, please look into the package rudel as that package was
also compatible with Gobby editor.

I do not say that you should make compatibility, that is not
necessary, it is better that we have it for Emacs only.

You may always try to install package:

M-x package-install-from-buffer

and you will see compiler warnings, it may be little different than if
you just use M-x emacs-lisp-byte-compile

In toplevel form:
crdt.el:77:1: Warning: Unused lexical variable ‘eol’
crdt.el:367:1: Warning: Unused lexical argument ‘key’
crdt.el:454:1: Warning: Unused lexical argument ‘crdt-buffer’
crdt.el:467:1: Warning: Unused lexical argument ‘k’
crdt.el:498:1: Error: Symbol’s function definition is void: crdt--with-buffer-name-make-body

There are some issues in the package for convention purposes:

;; Keywords: collaboration crdt

Keywords should only be those that you may find when using:

M-x finder-list-keywords

To verify if your keywords are alright, please use:

M-x checkdoc-package-keywords

It will give you errors.

People will start installing the package and it is good that you
increase version numbers, so that it is not always 0.0.0, at least
0.0.1 and so on, it could be 0.0.10 and 0.0.11 and I do not know if
there is some format for that, it is good to increase version numbers.

Commentary section shall later involve more instructions for users.

Docstrings are missing for some functions.

Items TODO, by my opinion:

- [ ] after M-x crdt-share-buffer, the prompt is as following:
  "Choose a server session (create if not exists): "

  However, there is nothing to be chosen.

  Thus:

  - [ ] if there is no server session to be chosen, let the user
    create new session, maybe something like: "New session name? "

  - "create if not exists" is imperative, but that asks user to create
    it. It has somehow vague meaning. The way how prompt looks like it
    gives me impression that some default session will be created if I
    just press enter. Is it maybe good to create default session
    automatically?

    Maybe username@host-number could be something good for new session
    name to be invented by default.

    This default value could be shown to user on the prompt, and user
    could just change it. See function docstring for
    read-from-minibuffer if you use that one

  - Other issue is that if I do press ENTER, it quits with the
    beep. That is not enough, it is better if you give message that
    crdt aborted.

    The workflow for me was: crdt-share-buffer, it tells me "create if
    not exists" and I may assume that it will be created by default, I
    press enter and then what? I can see "Quit" but I cannot know what
    really it means, did I quit session maybe? What did I quit?

    It is better to use prompt that will not let user go out of it by
    pressing enter. 

- [ ] I see that in M-x crdt-list-users there is one user name
  highlighted, I do not think that highlighting is necessary and if
  you include that, that you should not hard code highlighting as
  users may have such different themes. Additionally, is good to test
  it on console as well.

- [ ] crdt-connect should not fail because host name was not
  entered as it should not move to port prompt if host name is not
  entered. 

- [ ] my suggestion is that you try to share the buffer always, so I
  tried sharing this buffer for email to you. It gave me some
  errors. I was using M-x orgalist mode inside. My debugger opened up
  on error.

  Now, because there were errors, I could not save the buffer. 

  See this temporary video: https://gnu.support/files/tmp/2020-10-23/2020-10-23-10:54:21.ogv
  The video will self destruct after some weeks.

- [ ] M-x crdt-connect, now I get the screen of the session, I can
  see: Local Buffer in first column, what does that mean for me as
  remote user? Does it mean the remote buffer should become my local
  buffer? I do not know as user

- [ ] M-x crdt-connect, when entered into same list of buffers, I can
  see "Network Name" and under that column I can see the name of the
  file being edited. That is contradictory.

  I think that you should not call that Network as you have not asked
  user for Network name, you asked for Session name. Maybe you mean
  session, maybe file, now I do not know it.

- [ ] in the same list i see entry under Local Buffer "--" and this is
  also vague. If you wish to tell there is none buffer or similar,
  then you could say maybe "not yet connected".

- [ ] column width in the tabulated mode can be adjusted, you could
  assume safely that most users have at least 80 chars width. When I
  finally connect to the buffer, I see: crdt - mutt-pr... but  that is
  way to narrow, my file name was mutt-protected-1001-9722-56588427
  and more than that.

- [ ] Gobby editor also helps users choose a color. I see that cursor
  of other user is visible and it looks as block cursor. Some people
  will want to customize their own cursor to be like a line or else. I
  am just thinking as if you wish to show multiple cursors in one
  buffer which is also logical for multi user editing, then maybe you
  should assign to users various colors. Is that maybe why you
  highlighted the username in the list of buffers?

- [ ] I could not save the buffer shared, I tried killing it too. If I
  kill the buffer the connected user should maybe get informed and
  connection to that buffer should disconnect.

Debugger entered--Lisp error: (wrong-type-argument hash-table-p nil)
  puthash("mutt-protected-1001-9722-5658842758353437585" nil nil)
  (progn (puthash crdt--buffer-network-name nil (crdt--buffer-table)) (crdt--broadcast-maybe (crdt--format-message (cons 'cursor (cons crdt--buffer-network-name (cons (crdt--local-id) '(nil nil nil nil)))))) (if (eq (crdt--focused-buffer-name) crdt--buffer-network-name) (progn (crdt--broadcast-maybe (crdt--format-message (cons 'focus (cons (crdt--local-id) '...)))) (crdt--focused-buffer-name-setter nil))) (crdt--refresh-users-maybe))
  (if crdt--buffer-network-name (progn (puthash crdt--buffer-network-name nil (crdt--buffer-table)) (crdt--broadcast-maybe (crdt--format-message (cons 'cursor (cons crdt--buffer-network-name (cons (crdt--local-id) '...))))) (if (eq (crdt--focused-buffer-name) crdt--buffer-network-name) (progn (crdt--broadcast-maybe (crdt--format-message (cons 'focus (cons ... ...)))) (crdt--focused-buffer-name-setter nil))) (crdt--refresh-users-maybe)))
  crdt--kill-buffer-hook()
  kill-buffer("mutt-protected-1001-9722-5658842758353437585")
  funcall-interactively(kill-buffer "mutt-protected-1001-9722-5658842758353437585")
  #<subr call-interactively>(kill-buffer nil nil)
  apply(#<subr call-interactively> kill-buffer (nil nil))
  call-interactively@ido-cr+-record-current-command(#<subr call-interactively> kill-buffer nil nil)
  apply(call-interactively@ido-cr+-record-current-command #<subr call-interactively> (kill-buffer nil nil))
  call-interactively(kill-buffer nil nil)
  command-execute(kill-buffer)

-- 
Jean Louis

Re: TODO crdt-stop-session, selecting deleted buffer, fails to remove session from the list

[Eli Zaretskii]

> Date: Fri, 23 Oct 2020 11:10:28 +0300
> From: Jean Louis <bugs@gnu.support>
> Cc: emacs-tangents@gnu.org
> 
> I will post this to emacs-tangents, maybe other people wish to
> contribute to this package.

emacs-tangents is not for discussing Emacs development in general and
Emacs packages in particular.  That should happen on emacs-devel.
This list is for issues that are only tangentially related to Emacs.