Workshop to save M$ Windows users - help needed

Workshop to save M$ Windows users - help needed

[Eduardo Ochs]

Hi all,

I am preparing a workshop that will be a kind of an introduction to
Free Software to people who don't even know how programmers think but
who are curious to learn... It will happen by chat - initially on
Telegram, but I will try to teach people how to use rcirc - and my
main focus will be on:

  1. How in Elisp we can do lots of programs that are useful and that
     fit in just one or two lines,

  2. How we can share complete short programs with the other people in
     the workshop via chat. Parenthesis: I think that this will be the
     most mind-blowing part of the worshop for most of the
     participants... I am expecting an audience composed almost
     exclusively of M$ Windows users - "MWUs" from here onwards - and
     MWUs usually think that "programs" are huge beasts that only
     super-nerds that understand,

  3. How bigger programs can be built from smaller programs - we will
     play with some programs that are 5 to 10 lines long and made of
     several defuns (yaaay!!!) and that are easy to tinker with.

What I described above is the part of the workshop that I already know
how to teach, or to coordinate - "coordinate" because if everything
goes well then I'll have ten people speaking on the chat all at the
same time.

Now comes that part that I _DON'T_ know how to teach, and in which I
would like to ask for help.

I don't know how MWUs think. I don't know how they memorize actions
that are made of sequences of mouse clicks, I don't know how they take
notes, and I don't know how they can deal with programs without having
a clear mental model of what they do. I don't interact with them in
person close to computers often, but practically every time that they
see me using Emacs on my laptop they say "I don't have any idea how
you can use that", so they also don't understand how I use the
computer and how I think - it's reciprocal.

I talked to some MWUs who are interested in the workshop and they
don't have any idea of how to use terminals - they think that in order
to use a terminal one has to have a prodigious memory and prodigious
typing skills. Terminals are COMPLETELY alien to them, and for several
of them this is the main reason why they never tried to make their
computers dual-boot and to install a GNU/Linux in a partition.

I believe that I can use a part of the workshop to show them how we
can use something similar to the items (1), (2), and (3) above to run
Bash in a shell buffer in Emacs, and to follow a basic tutorial on
Bash... and that will make terminals far less alien to them.



So - HELP, PLEASE!...

I will only be able to do this "now let's do something similar but
with a tutorial for a Unix shell!" if I can get bulletproof-ish
instructions that they, the MWUs, can follow to install bash and wget
in their Windows machines - and the very few MWUs that I have contact
with and who are not super-hyper-mega-busy don't even know how to
think in terms of full paths... if they download a bash.exe they will
think that its "location" is something determined by a series of mouse
clicks, they have to struggle really hard to discover the full path to
that bash.exe, that will be something like "c:/foo/bar/bash.exe"...
and I am afraid that something like "make sure that the bash.exe is in
your PATH" would be almost impossibly hard to them...

Can we use this thread to discuss possible solutions to that?

Here's one idea. I can ask each one of them to run a sexp like this,

  (insert "\n\n" "MY NAME HERE" "\n" (getenv "PATH") "\n\n")

and send me the lines that it produces, and I can write a series of
eshell commands to each of the participants - that they will send to
eshell using something like this:

  http://angg.twu.net/IMAGES/2021-square-cube-3.png

(Obs: I am trying to learn Inkscape properly to produce lots of
figures like the one above... that one, that was my first, took me,
huh, I don't know, one hour?...)

  Thanks in advance!!!

    Eduardo Ochs
    http://angg.twu.net/#eev

Re: Workshop to save M$ Windows users - help needed

[Jean Louis]

* Eduardo Ochs <eduardoochs@gmail.com> [2021-10-03 07:28]:
> Hi all,
> 
> I am preparing a workshop that will be a kind of an introduction to
> Free Software to people who don't even know how programmers think but
> who are curious to learn... It will happen by chat - initially on
> Telegram, but I will try to teach people how to use rcirc - and my
> main focus will be on:

OK, though here is short opinion:

- if you advertise channel or anything through Telegram it is vendor
  lock network, promoting the network, not your own service or
  organization. If anything happens to Telegram, you will lose
  connections. It is centralized system.

- to make your own XMPP server is not much, it could be US $2.50 per
  month on Vultr VPS

- to have your own domain for 1 year is also not much, I will gladly
  assign you subdomain for ever, at least I got XMPP.CLUB, I can give
  you eev.xmpp.club if you wish or anything similar.

- if you wish to have your own video chat, that is also possible, why
  not use Jami, https://jami.net and there are other solutions.

>   1. How in Elisp we can do lots of programs that are useful and that
>      fit in just one or two lines,

It is great idea to promote it. Though one or two lines will become
very very long...

>   2. How we can share complete short programs with the other people in
>      the workshop via chat.

If you are promoting Elisp, then best solution is to use your own XMPP
server such as Prosody installed on US $2.50 and the program named
`jabber.el' that enables Emacs to communicate by XMPP, which I use
everyday to coordinate our teams.

That way you will be sharing communication straight Emacs and be able
even to evaluate straight in the Jabber/XMPP buffers.

> What I described above is the part of the workshop that I already know
> how to teach, or to coordinate - "coordinate" because if everything
> goes well then I'll have ten people speaking on the chat all at the
> same time.

I just say, XMPP through Emacs is better for text transmission and
speaking could go unlimited with your Mumle server. URL:
//www.mumble.info/

To install server software is easy, and I can help.

Client software is easy and people can connect easy. It has better
latency, thus better speech and connection then other solutions I have
tried. 

> I don't know how MWUs think.

I suggest not to label people with "MWUs".

Everybody thinks different, is not even useful to try to analyze it,
and there are many programmers on Windows including Emacs Lisp users.

You want to teach people, provide lessons, don't worry how they think,
there will be interested people, that is what matters.

> I don't know how they memorize actions that are made of sequences of
> mouse clicks, I don't know how they take notes, and I don't know how
> they can deal with programs without having a clear mental model of
> what they do. I don't interact with them in person close to
> computers often, but practically every time that they see me using
> Emacs on my laptop they say "I don't have any idea how you can use
> that", so they also don't understand how I use the computer and how
> I think - it's reciprocal.

I had computer club for 2 times, and it will now be third time. You
can teach ANYBODY with or without previous computer experience in
computing. That is up to you to kind of align to the other person. I
have been teaching people who have never see a computer before and
came from agricultural fields straight to club. I remember the funny
typing movements with the fingers first up and then down, like a hawk
that is catching a mouse on the field. After few weeks the guy was
programming. That is what matters. We had children of 7-8 years
programming, and of 11-12 years programming in machine language. Don't
complicate. You can teach people anything.

Larger problem I see simply in the general environment of people such
as their country and culture of learning, reading books and
similar. If there are a lot of books in the generally spoken language
in the country, and there is culture of reading and learning, then you
should not have any problems.

There countries where there are almost no books, people don't keep
books at home, don't read, and don't teach their children to read. 

> I talked to some MWUs who are interested in the workshop and they
> don't have any idea of how to use terminals - they think that in order
> to use a terminal one has to have a prodigious memory and prodigious
> typing skills. Terminals are COMPLETELY alien to them, and for several
> of them this is the main reason why they never tried to make their
> computers dual-boot and to install a GNU/Linux in a partition.

Instead of analyzing hypothetically, simply ask if they wish to try it
and help with the installation. 

I never encountered your problems and I have helped many people switch
to GNU/Linux. It was in Germany.

> I believe that I can use a part of the workshop to show them how we
> can use something similar to the items (1), (2), and (3) above to run
> Bash in a shell buffer in Emacs, and to follow a basic tutorial on
> Bash... and that will make terminals far less alien to them.

Ask yourself "why"? Do they really need a terminal or Bash? I don't
think so. 

If you teach Emacs, use TUTORIAL as your starting point, but even more
fundamental is knowing how to use keys on the keyboard. Majority of
people will get stuck with proper usage of special keys.

1. Talk about proper usage of keyboard and special keys;

2. Going through Emacs Tutorial;

3. Use Emacs Lisp introduction and go through it;


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Workshop to save M$ Windows users - help needed

[Eduardo Ochs]

On Sun, 3 Oct 2021 at 02:48, Jean Louis <bugs@gnu.support> wrote:
>
> (...)


Hi Jean-Louis,

my experience is that people here don't mind spending weeks or months
trying to learn how to use some program _IF_ "everybody" - for some
value of "everybody" - keeps repeating that that program is incredibly
important and useful, and that is it "easy" and that "there are lots
of tutorials about it on the internet"...

The text that I wrote explaining my workshop is hard to translate to
English, so let me just retell in English what some parts of it say.

I start the text by describing a situation that practically everybody
has been through: we see that one of our friends has included some
amazing graphics in one of his assignments, and we ask how we can
learn how to do graphics like that ourselves. He answers something
like: "I used Python and MatPlotLib! They are super easy to learn!
There are lots of tutorials on the internet!" - and then we spend the
next weekend at home trying to learn Python and MatPlotLib from the
tutorials that we found on the internet, but we progress very little.
We ask more people for more hints and help, and the answers are always
in that same form: "use the programs/libraries such and such", "it's
easy", "there are tutorials on the internet"...

Then I explain that there are parts of the Free Software World in
which people have other ways to help one another, and in which if you
ask "how do I do such and such?" very often people will answer with
snippets that do something closely related to what you want, and they
will say "does this help?" - and by reading and understanding those
snippets you will learn a lot, and very often you will be able to
modify them to get code that does what you want.

My main intention with that workshop will be to introduce people to
that world - in which people share short programs instead of just
sharing names of programs and libraries and saying "it's easy".

I also explained in the text that usually we go to workshops expecting
to learn something "useful", but we don't notice that we have a
certain rigid notion of "usefulness" that we never thought much about
it. Probably what is going to happen in this workshop is that people
will start to understand other notions of "usefulness", and for
programmers often something is "useful" when it is a building block
can be used in many ways. And my criteria for choosing what to present
on this workshop will be not "usefulness" but "elegance" and "fun" - I
will present some things that are easy to understand but that have
deep ideas inside, and that _some_ of my programmer friends consider
that are "elegant" and "fun".



You said:

  - if you advertise channel or anything through Telegram
    it is vendor lock network, promoting the network, not
    your own service or organization. If anything happens
    to Telegram, you will lose connections. It is
    centralized system.

The plan is to use Telegram - that many of the students already know
how to use - to start, and then switch to rcirc... suppose that this
means that we will use Telegram for 1h30, and then abandon it. Does
that really look very bad to you?

You said:

  Ask yourself "why"? Do they really need a terminal or
  Bash? I don't think so.

  If you teach Emacs, use TUTORIAL as your starting point,
  but even more fundamental is knowing how to use keys on
  the keyboard. Majority of people will get stuck with
  proper usage of special keys.

  1. Talk about proper usage of keyboard and special keys;

  2. Going through Emacs Tutorial;

  3. Use Emacs Lisp introduction and go through it;

"Emacs" and "Unix" have several meanings - to me -, and I am
interested in the meaning in which they are "anti-black-boxes"
and are environments that encourage people to help one another
by sharing snippets of code.

  [[]],
    Eduardo Ochs
    http://angg.twu.net/#eev

Re: Workshop to save M$ Windows users - help needed

[Eli Zaretskii]

> From: Eduardo Ochs <eduardoochs@gmail.com>
> Date: Sun, 3 Oct 2021 01:26:59 -0300
> 
>   1. How in Elisp we can do lots of programs that are useful and that
>      fit in just one or two lines,
> 
>   2. How we can share complete short programs with the other people in
>      the workshop via chat. Parenthesis: I think that this will be the
>      most mind-blowing part of the worshop for most of the
>      participants... I am expecting an audience composed almost
>      exclusively of M$ Windows users - "MWUs" from here onwards - and
>      MWUs usually think that "programs" are huge beasts that only
>      super-nerds that understand,
> 
>   3. How bigger programs can be built from smaller programs - we will
>      play with some programs that are 5 to 10 lines long and made of
>      several defuns (yaaay!!!) and that are easy to tinker with.
> 
> What I described above is the part of the workshop that I already know
> how to teach, or to coordinate - "coordinate" because if everything
> goes well then I'll have ten people speaking on the chat all at the
> same time.
> 
> Now comes that part that I _DON'T_ know how to teach, and in which I
> would like to ask for help.
> 
> I don't know how MWUs think. I don't know how they memorize actions
> that are made of sequences of mouse clicks, I don't know how they take
> notes, and I don't know how they can deal with programs without having
> a clear mental model of what they do. I don't interact with them in
> person close to computers often, but practically every time that they
> see me using Emacs on my laptop they say "I don't have any idea how
> you can use that", so they also don't understand how I use the
> computer and how I think - it's reciprocal.
> 
> I talked to some MWUs who are interested in the workshop and they
> don't have any idea of how to use terminals - they think that in order
> to use a terminal one has to have a prodigious memory and prodigious
> typing skills. Terminals are COMPLETELY alien to them, and for several
> of them this is the main reason why they never tried to make their
> computers dual-boot and to install a GNU/Linux in a partition.
> 
> I believe that I can use a part of the workshop to show them how we
> can use something similar to the items (1), (2), and (3) above to run
> Bash in a shell buffer in Emacs, and to follow a basic tutorial on
> Bash... and that will make terminals far less alien to them.

I don't understand why you need to tell them to install Bash.
Installing Bash means installing MSYS2, which comes with lots of
ballast most people won't need.  In addition, Bash is a totally alien
beast for your audience, even if they are familiar with the Windows'
own shell, either cmd.exe or PowerShell.  You are raising the bar very
highly without any justification.

If your main goal is to teach them Emacs as the platform to do these
small jobs, my suggestion is to use Eshell.  Even that is already
problematic, because for some jobs they will have to install ports of
GNU tools: Grep, Diff, etc.  But at least they could limit themselves
to installing a small number of relatively small packages, and won't
need to deal with the mess of MSYS2 executables vs native Windows
(a.k.a. MinGW64) executables, and won't need Bash.

Re: Workshop to save M$ Windows users - help needed

[Tomas Hlavaty]

On Sun 03 Oct 2021 at 01:26, Eduardo Ochs <eduardoochs@gmail.com> wrote:
> I talked to some MWUs who are interested in the workshop and they
> don't have any idea of how to use terminals - they think that in order
> to use a terminal one has to have a prodigious memory and prodigious
> typing skills. Terminals are COMPLETELY alien to them, and for several
> of them this is the main reason why they never tried to make their
> computers dual-boot and to install a GNU/Linux in a partition.

Maybe do not scare them with the word terminal.
Call it the black window.
Use that unique color to your advantage.
They will remember it and always find it.
Show them what up and down arrows do.
This worked with my mum.

Re: Workshop to save M$ Windows users - help needed

[Eduardo Ochs]

On Sun, 3 Oct 2021 at 06:38, Eli Zaretskii <eliz@gnu.org> wrote:
>
> I don't understand why you need to tell them to install Bash.
> Installing Bash means installing MSYS2, which comes with lots of
> ballast most people won't need.  In addition, Bash is a totally alien
> beast for your audience, even if they are familiar with the Windows'
> own shell, either cmd.exe or PowerShell.  You are raising the bar very
> highly without any justification.
>
> If your main goal is to teach them Emacs as the platform to do these
> small jobs, my suggestion is to use Eshell.  Even that is already
> problematic, because for some jobs they will have to install ports of
> GNU tools: Grep, Diff, etc.  But at least they could limit themselves
> to installing a small number of relatively small packages, and won't
> need to deal with the mess of MSYS2 executables vs native Windows
> (a.k.a. MinGW64) executables, and won't need Bash.

Ok, let's forget bash, then!
Eshell should be enough.

What is a good way for installing wget and a program from
unzipping/untgzipping files? Which un(tg)zipper do you recommend? I
remember vaguely using something called 7z ages ago...

Also, what is a good way to make sure that these programs are in the
PATH? If each student can send me their original PATH and the full
paths to these programs I can produce a sexp like this for each one...

  (setenv "PATH"
   (concat
    "/path/to/wget:"
    "/path/to/7z:"
    "/home/edrx/bin:/home/edrx/.elan/bin:/home/edrx/.local/bin:/home/edrx/.cabal/bin:/home/edrx/.ghcup/bin:/usr/local/texlive/2021/bin/x86_64-linux:/home/edrx/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:.:/home/edrx/EXPECT:/home/edrx/TCL:/home/edrx/pkg/bin")
   )

  Thanks in advance =),
    E.

Re: Workshop to save M$ Windows users - help needed

[Eli Zaretskii]

> From: Eduardo Ochs <eduardoochs@gmail.com>
> Date: Sun, 3 Oct 2021 07:19:16 -0300
> Cc: help-gnu-emacs <help-gnu-emacs@gnu.org>
> 
> What is a good way for installing wget and a program from
> unzipping/untgzipping files? Which un(tg)zipper do you recommend? I
> remember vaguely using something called 7z ages ago...

Do you really need that?  The Windows Explorer can unzip files just
fine, and a browser or Emacs itself can be used to fetch files.

> Also, what is a good way to make sure that these programs are in the
> PATH? If each student can send me their original PATH and the full
> paths to these programs I can produce a sexp like this for each one...
> 
>   (setenv "PATH"
>    (concat
>     "/path/to/wget:"
>     "/path/to/7z:"
>     "/home/edrx/bin:/home/edrx/.elan/bin:/home/edrx/.local/bin:/home/edrx/.cabal/bin:/home/edrx/.ghcup/bin:/usr/local/texlive/2021/bin/x86_64-linux:/home/edrx/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:.:/home/edrx/EXPECT:/home/edrx/TCL:/home/edrx/pkg/bin")
>    )

It's best not to go there at all.  Changing PATH from within Emacs is
asking for trouble, because the system outside Emacs doesn't know
about that, and changing PATH system-wide requires going to system
properties and making changes there (which sometimes might mean
privileges normal users don't have).  Try avoiding that, is my advice.

Re: Workshop to save M$ Windows users - help needed

[Eduardo Ochs]

On Sun, 3 Oct 2021 at 07:41, Eli Zaretskii <eliz@gnu.org> wrote:
>
> Do you really need that?  The Windows Explorer can unzip files just
> fine, and a browser or Emacs itself can be used to fetch files.


Hi Eli,

Let me start by showing some kinds of useful one-liners.


  1. (find-eev-quick-intro "2. Evaluating Lisp" "M-0 M-e")
     (find-eev-quick-intro "3.1. Non-elisp hyperlinks" "M-x brff")
     (find-emacs-keys-intro "5. Undoing")

     The first sexp above opens the sandboxed tutorial called
     `find-eev-quick-intro', searches for the first occurrence of the
     string "2. Evaluating Lisp" in it, and then searches for the
     first occurrence of the string "M-0 M-e" after that.



  2. (find-firefox-page "http://foo/bar/Coetzee99.pdf" 4)
     (find-firefox-page              "~/Coetzee99.pdf" 4)

     The two sexps above use Firefox as a PDF viewer to open a remote
     PDF and a local PDF on page 4. In low-level terms what they do is:

     (find-bgprocess '("firefox"    "http://foo/bar/Coetzee99.pdf#page=4"))
     (find-bgprocess '("firefox" "file:///home/edrx/Coetzee99.pdf#page=4"))



  3. (find-testblsvideo "2:33" "f8 here starts a new Lua interpreter")

     This sexp plays the video about test blocks starting from 2:33.
     Let me pretend here that I would have time enough in the workshop
     to show that it has two "natural" definitions. Its first natural
     definition would use the copy of that video at youtube, and would
     use Firefox to open this URL:

        http://www.youtube.com/watch?v=fpsF_M55W4o#t=2m33s

     Its second natural definition supposes that we have downloaded
     the .mp4 file of that video, that is here,

        http://angg.twu.net/eev-videos/2021-test-blocks.mp4

     to some hardcoded location, like:

        /tmp/foo/bar/2021-test-blocks.mp4

     and it supposes that everyone in the workshop has the mpv video
     player installed. In low-level terms, what this second definition
     runs is:

       (find-bgprocess '("mpv" "/tmp/foo/bar/2021-test-blocks.mp4"
                         "--start=2:33" "--fs" "--osd-level=2"))



  4. (find-pdf-text "~/Coetzee99.pdf" (+ -110 127) "wrong thoughts")

     This is a variant of the sexps in item 2. It is not trivial to
     make `find-pdf-text' work on Windows, but that function is a
     fixture of my workshops on LaTeX. Let me pretend that I would be
     able to present it, just because it is a very nifty function and
     it will be good karma to give that function a bit more of
     visibility.

     That sexp converts the file ~/Coetzee99.pdf to text by running:

       pdftotext -layout -enc Latin1 /home/edrx/Coetzee99.pdf -

     then puts the output in a temporary buffer, finds the page 17,
     i.e., (+ -110 127), in it by counting formfeeds, and searches for
     the first occurrence of the string "wrong thoughts" in that
     buffer starting from the beginning of page 17.



So, consider again these sexps:

  (find-eev-quick-intro "2. Evaluating Lisp" "M-0 M-e")
  (find-eev-quick-intro "3.1. Non-elisp hyperlinks" "M-x brff")
  (find-emacs-keys-intro "5. Undoing")
  (find-firefox-page "http://foo/bar/Coetzee99.pdf" 4)
  (find-firefox-page              "~/Coetzee99.pdf" 4)
  (find-testblsvideo "2:33" "f8 here starts a new Lua interpreter")
  (find-pdf-text "~/Coetzee99.pdf" (+ -110 127) "wrong thoughts")

all of them are easy to understand, easy to use - if you have the
right files and if you know how to use `M-e', that is roughly
equivalent to `C-e C-x C-e' -, easy to modify in useful ways (by
changing their arguments), easy to save in your notes with cut and
paste, and easy to share by chat.

Two of these sexps need files in hardcoded places - I used
~/Coetzee99.pdf and /tmp/foo/bar/2021-test-blocks.mp4 in the examples
above, but many functions in eev use a convention, explained here,

  http://angg.twu.net/eev-intros/find-psne-intro.html

in which the name of the local copy is derived from the URL like this,

      http://www.gnu.org/software/emacs/emacs-paper.html
  -> $S/http/www.gnu.org/software/emacs/emacs-paper.html

and the environment variable S is $HOME/snarf. Also, if we put the
point on the url below

      http://www.gnu.org/software/emacs/emacs-paper.html

and type M-x brep we get a temporary buffer containing something like
this,


• (eepitch-shell)
• (eepitch-kill)
• (eepitch-shell)
  mkdir -p $S/http/www.gnu.org/software/emacs/
  cd       $S/http/www.gnu.org/software/emacs/
  wget -nc 'http://www.gnu.org/software/emacs/emacs-paper.html'


So several things in the workshop will be much easier if everybody has
wget in the path. An un(tg)zipper is not very important - I only
realized this after my previous e-mail - but wget is.

  [[]],
    Eduardo Ochs
    http://angg.twu.net/#eev

Re: Workshop to save M$ Windows users - help needed

[Tomas Hlavaty]

On Sun 03 Oct 2021 at 16:15, Eduardo Ochs <eduardoochs@gmail.com> wrote:
> On Sun, 3 Oct 2021 at 07:41, Eli Zaretskii <eliz@gnu.org> wrote:
>> Do you really need that?  The Windows Explorer can unzip files just
>> fine, and a browser or Emacs itself can be used to fetch files.
>   mkdir -p $S/http/www.gnu.org/software/emacs/
>   cd       $S/http/www.gnu.org/software/emacs/
>   wget -nc 'http://www.gnu.org/software/emacs/emacs-paper.html'
> [...]
> So several things in the workshop will be much easier if everybody has
> wget in the path.

Emacs should be able to download the files even on Windows without
installing wget, e.g. this should work:

(eww "http://www.gnu.org/software/emacs/emacs-paper.html")

There is a variable url-cache-directory suggesting that caching can
happen transparently.  Maybe you could even supply your own
url-cache-creation-function.  Or figure out what functions you could
reuse to simulate wget and your caching strategy in pure Emacs more
closely.

I know you want to do it manually, but why bother beginners with that?

Re: Workshop to save M$ Windows users - help needed

[Eduardo Ochs]

On Sun, 3 Oct 2021 at 16:44, Tomas Hlavaty <tom@logand.com> wrote:
>
> Emacs should be able to download the files even on Windows without
> installing wget, e.g. this should work:
>
> (eww "http://www.gnu.org/software/emacs/emacs-paper.html")
>
> There is a variable url-cache-directory suggesting that caching can
> happen transparently.  Maybe you could even supply your own
> url-cache-creation-function.  Or figure out what functions you could
> reuse to simulate wget and your caching strategy in pure Emacs more
> closely.
>
> I know you want to do it manually, but why bother beginners with that?


Hi Tomas,

I've tried to implement something like what you are proposing. It
seems that the right tool for that, at least for the simplest
cases, is `url-retrieve-synchronously'. It is documented here:

  (find-node "(url)Retrieving URLs")
  (find-node "(url)Retrieving URLs" "url-retrieve-synchronously")
  (find-node "(url)Dealing with HTTP documents")

Straight.el uses `url-retrieve-synchronously' here:

  https://github.com/raxod502/straight.el#getting-started

Try this:

  (find-ebuffer
   (url-retrieve-synchronously
    "https://raw.githubusercontent.com/raxod502/straight.el/develop/install.el"
    'silent 'inhibit-cookies)
   "\n\n")

The "\n\n" will put the point after the response headers.

I thought that it would be trivial to write a "fake wget" for
eshell using `url-retrieve-synchronously'. The part of adding a
new builting seems to be easy, see:

  (find-node "(eshell)Built-ins" "eshell/ls")
  (find-eshellfile "")
  (find-eshellgrep "grep --color=auto -nH --null -e eshell/ *.el")

but I couldn't get the error checking right, and at some point I
stopped trying. Then I saw this thread,

  https://lists.gnu.org/archive/html/emacs-devel/2021-03/threads.html#01359

and it convinced me that using a real wget would be the right
thing to do.

I have some code for playing with url-retrieve here, if you'd
like to try:

                     http://angg.twu.net/elisp/url-retrieve-test.el
                     http://angg.twu.net/elisp/url-retrieve-test.el.html
  (find-wgeta-elisp "http://angg.twu.net/elisp/url-retrieve-test.el")

  Cheers,
    Eduardo Ochs
    http://angg.twu.net/#eev

Re: Workshop to save M$ Windows users - help needed

[Tomas Hlavaty]

On Mon 04 Oct 2021 at 00:06, Eduardo Ochs <eduardoochs@gmail.com> wrote:
> On Sun, 3 Oct 2021 at 16:44, Tomas Hlavaty <tom@logand.com> wrote:
>>
>> Emacs should be able to download the files even on Windows without
>> installing wget, e.g. this should work:
>>
>> (eww "http://www.gnu.org/software/emacs/emacs-paper.html")
>>
>> There is a variable url-cache-directory suggesting that caching can
>> happen transparently.  Maybe you could even supply your own
>> url-cache-creation-function.  Or figure out what functions you could
>> reuse to simulate wget and your caching strategy in pure Emacs more
>> closely.
>>
>> I know you want to do it manually, but why bother beginners with that?
>
>
> Hi Tomas,
>
> I've tried to implement something like what you are proposing. It
> seems that the right tool for that, at least for the simplest
> cases, is `url-retrieve-synchronously'. It is documented here:
>
>   (find-node "(url)Retrieving URLs")
>   (find-node "(url)Retrieving URLs" "url-retrieve-synchronously")
>   (find-node "(url)Dealing with HTTP documents")
>
> Straight.el uses `url-retrieve-synchronously' here:
>
>   https://github.com/raxod502/straight.el#getting-started
>
> Try this:
>
>   (find-ebuffer
>    (url-retrieve-synchronously
>     "https://raw.githubusercontent.com/raxod502/straight.el/develop/install.el"
>     'silent 'inhibit-cookies)
>    "\n\n")
>
> The "\n\n" will put the point after the response headers.
>
> I thought that it would be trivial to write a "fake wget" for
> eshell using `url-retrieve-synchronously'. The part of adding a
> new builting seems to be easy, see:
>
>   (find-node "(eshell)Built-ins" "eshell/ls")
>   (find-eshellfile "")
>   (find-eshellgrep "grep --color=auto -nH --null -e eshell/ *.el")
>
> but I couldn't get the error checking right, and at some point I
> stopped trying. Then I saw this thread,
>
>   https://lists.gnu.org/archive/html/emacs-devel/2021-03/threads.html#01359
>
> and it convinced me that using a real wget would be the right
> thing to do.
>
> I have some code for playing with url-retrieve here, if you'd
> like to try:
>
>                      http://angg.twu.net/elisp/url-retrieve-test.el
>                      http://angg.twu.net/elisp/url-retrieve-test.el.html
>   (find-wgeta-elisp "http://angg.twu.net/elisp/url-retrieve-test.el")
>
>   Cheers,
>     Eduardo Ochs
>     http://angg.twu.net/#eev

I see, url-retrieve-synchronously is a beast.  You could write something
simpler and keep track of the status.  Something like:

;;; -*- lexical-binding: t -*-

(defun url-retrieve-synchronously2 (url filename)
  (let ((b (find-file filename))
        z
        (again t))
    (cl-flet ((save (status done)
                    ;;(message "save %s %s" status done)
                    (setq z status)
                    (if done
                        (setq again nil)
                      (when status
                        (with-current-buffer b
                          (let ((coding-system-for-write 'raw-text-unix))
                            (write-region (point-min) (point-max) filename)))))))
      (with-current-buffer b
        (url-retrieve url #'save '(t)))
      (while again
        (save nil nil)
        (sleep-for 1))
      ;;(message "done")
      z)))

(url-retrieve-synchronously2 "https://logand.com" "/tmp/e2")
(url-retrieve-synchronously2 "https://logand.com1" "/tmp/e2")

This works for me (saves whole http response), returns plist and saves
the data into file on success, otherwise on failure returns nil.

Anyway, I was trying to suggest something even simpler: use eww and
customize it so that it caches the downloaded files.  This is what you
are doing in shell manually.

Or even better, bind for example M-f to ffap so that people can simply
press M-f on a url and Emacs will take them there.

Re: Workshop to save M$ Windows users - help needed

[Tomas Hlavaty]

url-retrieve-synchronously2 had a bug

this works and should dispose the nework buffer properly:

(defun url-retrieve-synchronously7 (url filename)
  (let (z (again t))
    (url-retrieve url
                  (lambda (status)
                    (setq again nil)
                    (when status
                      (setq z status)
                      (let ((coding-system-for-write 'raw-text-unix))
                        (write-region (point-min) (point-max) filename)))
                    (kill-buffer)))
    (while again
      (sleep-for 1))
    z))

(url-retrieve-synchronously7 "https://logand.com" "/tmp/e7")
(url-retrieve-synchronously7 "https://logand.com1" "/tmp/e7")
/tmp/e7

There is a comment in url-retrieve-synchronously about sleep-for and
similar suggesting that that might not always work but it works for me.
Not sure about windows.

I think url-retrieve-synchronously should not ignore the status but
return it instead.

What other issue did you have with url-retrieve-synchronously apart from
not being able to detect errors?  Your link to the thread discussed only
preference for libcurl.  I did not find what is wrong with
url-retrieve-synchronously there.  Another thing wrong with that is
probably that the network buffer is returned but iirc buffers are not
garbage collected, or something like that.  That's why I put kill-buffer
there.

Re: Workshop to save M$ Windows users - help needed

[Eli Zaretskii]

> From: Eduardo Ochs <eduardoochs@gmail.com>
> Date: Sun, 3 Oct 2021 16:15:11 -0300
> Cc: help-gnu-emacs <help-gnu-emacs@gnu.org>
> 
>       http://www.gnu.org/software/emacs/emacs-paper.html
> 
> and type M-x brep we get a temporary buffer containing something like
> this,
> 
> 
> • (eepitch-shell)
> • (eepitch-kill)
> • (eepitch-shell)
>   mkdir -p $S/http/www.gnu.org/software/emacs/
>   cd       $S/http/www.gnu.org/software/emacs/
>   wget -nc 'http://www.gnu.org/software/emacs/emacs-paper.html'
> 
> 
> So several things in the workshop will be much easier if everybody has
> wget in the path. An un(tg)zipper is not very important - I only
> realized this after my previous e-mail - but wget is.

I still don't think I understand.  Emacs is perfectly capable of
fetching URLs, so why would you need wget?  If you base this on wget,
you will need to cope with complexities that you had better avoided:
how to download a program, where to find it, how to install it so it's
on PATH, etc. etc.  Please believe me that arranging for a simple Lisp
command that fetches a URL and saves it to a file is orders of
magnitude simpler.

Re: Workshop to save M$ Windows users - help needed

[Eduardo Ochs]

On Mon, 4 Oct 2021 at 15:51, Eli Zaretskii <eliz@gnu.org> wrote:
>
> I still don't think I understand.  Emacs is perfectly capable of
> fetching URLs, so why would you need wget?  If you base this on wget,
> you will need to cope with complexities that you had better avoided:
> how to download a program, where to find it, how to install it so it's
> on PATH, etc. etc.  Please believe me that arranging for a simple Lisp
> command that fetches a URL and saves it to a file is orders of
> magnitude simpler.

Hi Eli,

I have a ton of code that uses wget. I don't have an elisp version of
wget yet, but I am trying to write one. I am working on two things at
the same time now:

  1. a function eshell/fakewget that will run a minimalist fake wget
     written in Lisp,

  2. a way to download a wget.exe that works that can used in eshell.

I prefer to have both of them instead of just one.

  Cheers,
    Eduardo Ochs

Re: Workshop to save M$ Windows users - help needed

[Eduardo Ochs]

On Mon, 4 Oct 2021 at 15:29, Tomas Hlavaty <tom@logand.com> wrote:
>
> url-retrieve-synchronously2 had a bug
>
> this works and should dispose the nework buffer properly:


Hi Tomas (and all),

Here are some updates on the problem of using eev to install wget.exe
on Windows machines.  Remember that I have good reasons to suppose the
most of the people who will participate in my workshop have never used
terminals and don't even know well enough what is a directory.


1. PowerShell
=============
Someone told me to take a look at PowerShell.  Apparently it has a
built-in wget, and it is somewhat multi-platform-ish.  I installed it
on my Debian box.  Its main executable is called pwsh.  Pwsh doesn't
run well on comint buffers - it says that the terminal is not powerful
enough.  It works well on vterm and ansi-term, and so I added support
for ansi-term to eev, and I defined a function eepitch-pwsh that uses
ansi-term.  I tested its built-in wget on my Debian, and it works
nice.  I found a friend - who uses Windows but who has never used
terminals - who volunteered to help with the tests.  It turns out that
on Windows the PowerShell executable is called powershell, not pwsh,
and it doesn't run well in an ansi-term, but it runs in comint.  We
tried the scripts that I prepared and they did't work - the wget
built-in in my friend's PowerShell behaved in a way totally different
from what I expected, and when we used it to download a 400KB file it
saved to disk something that had 2MB.  Then my friend found this:

https://superuser.com/questions/362152/native-alternative-to-wget-in-windows-powershell/758510#758510

And then we decided that using PowerShell would be bad karma, and we
gave up.



2. url-retrieve-synchronously
=============================
My friend was having a lot of fun executing sexps, so I decided to use
`url-retrieve-synchronously' to implement a kind of very primitive
fake wget in elisp - obs: I haven't tried your code from

  https://lists.gnu.org/archive/html/help-gnu-emacs/2021-10/msg00073.html
  https://lists.gnu.org/archive/html/help-gnu-emacs/2021-10/msg00075.html

yet, but I will soon! - and I got this:

  http://angg.twu.net/eev-current/eev-plinks.el#find-urlretrieve
  (find-wget-elisp "http://angg.twu.net/eev-current/eev-plinks.el"
"ee-very-primitive-wget0")



3. wget.exe
===========
There's a wget for Windows here:

  http://gnuwin32.sourceforge.net/packages/wget.htm
  http://downloads.sourceforge.net/gnuwin32/wget-1.11.4-1-bin.zip

• (eepitch-shell)
• (eepitch-kill)
• (eepitch-shell)

  rm -Rv /tmp/wget/
  mkdir  /tmp/wget/
  cd     /tmp/wget/
  wget http://downloads.sourceforge.net/gnuwin32/wget-1.11.4-1-bin.zip
  unzip wget-1.11.4-1-bin.zip

  file bin/wget.exe
  # bin/wget.exe: PE32 executable (console) Intel 80386 (stripped to
  # external PDB), for MS Windows

I unpacked the .zip in my machine, uploaded the wget.exe to a
temporary place in my homepage, and my friend downloaded it with my
function `ee-very-primitive-wget0'.  The download worked - the file
was not corrupted, it seems - but when he tried to execute that
wget.exe he got a message saying that that file was not compatible
with the version of Windows that we was using - which is "Microsoft
Windows 11 Home Single Language".


That's what we did today. More news soon...

  Cheers,
    Eduardo Ochs
    http://angg.twu.net/#eev

Re: Workshop to save M$ Windows users - help needed

[Emanuel Berg via Users list for the GNU Emacs text editor]

Eduardo Ochs wrote:

> Here are some updates on the problem of using eev to install
> wget.exe on Windows machines. Remember that I have good
> reasons to suppose the most of the people who will
> participate in my workshop have never used terminals and
> don't even know well enough what is a directory.
>
> 1. PowerShell
> =============
> Someone told me to take a look at PowerShell. Apparently it
> has a built-in wget, and it is somewhat multi-platform-ish.
> I installed it on my Debian box. Its main executable is
> called pwsh. Pwsh doesn't run well on comint buffers - it
> says that the terminal is not powerful enough. It works well
> on vterm and ansi-term, and so I added support for ansi-term
> to eev, and I defined a function eepitch-pwsh that uses
> ansi-term. [...]

Meta: ?

What's this new style of posting you guys do? Donald Knuth all
over? (Not that I know/saw it, just heard it. There was a long
discussion recently, don't look that up, embarrassing, heh :))

But I don't mind, on the contrary as they say in the sweet
science, "styles make fights" ... there is a guy in
gmane.emacs.erc.general who also does it! He is ... either
almost scary or very impressive ...

Here is also a workshop BTW, inspired by Emacs. Or did I came
to Emacs because of "the workshop" inside me, embryotic at
that time? Because you can take the tiger out of the jungle
but etc ...

  https://dataswamp.org/~incal/ums/

I've been telling the folks there it doesn't matter what
_order_ you do good things. They are idempotent! No ... that's
when nothing happens the second time you do it ... they are
a transaction? related but not it. they are/can be done in
parallel? but they can't, sometimes they require the same
tools and we have a finite number of those as you see -
especially in the machine shop ... but these things tend to
multiply when you get it organized and in one place (but teen
girls can't have the same effect with a box with
chronologically sorted LOVE LETTERS <3 )... anyway, just do it
if it makes sense, and don't worry about the order!
BUT DESPITE telling this my most important message to the
masses over and over - yes, I did take those photos AFTER
I cleaned and organized it :D

-- 
underground experts united
https://dataswamp.org/~incal

Re: Workshop to save M$ Windows users - help needed

[Eli Zaretskii]

> From: Eduardo Ochs <eduardoochs@gmail.com>
> Date: Wed, 6 Oct 2021 01:50:00 -0300
> Cc: help-gnu-emacs <help-gnu-emacs@gnu.org>
> 
>   wget http://downloads.sourceforge.net/gnuwin32/wget-1.11.4-1-bin.zip
>   unzip wget-1.11.4-1-bin.zip
> 
>   file bin/wget.exe
>   # bin/wget.exe: PE32 executable (console) Intel 80386 (stripped to
>   # external PDB), for MS Windows

That's a 32-bit build of (an old version) of wget.

> I unpacked the .zip in my machine, uploaded the wget.exe to a
> temporary place in my homepage, and my friend downloaded it with my
> function `ee-very-primitive-wget0'.  The download worked - the file
> was not corrupted, it seems - but when he tried to execute that
> wget.exe he got a message saying that that file was not compatible
> with the version of Windows that we was using - which is "Microsoft
> Windows 11 Home Single Language".

I guess the user will have to turn on some backward compatibility
settings for that to work.

Alternatively, download a newer wget.exe.  I think the wget project
publishes a pre-built binary?

Re: Workshop to save M$ Windows users - help needed

[Eduardo Ochs]

On Mon, 4 Oct 2021 at 15:29, Tomas Hlavaty <tom@logand.com> wrote:
>
> url-retrieve-synchronously2 had a bug
>
> this works and should dispose the nework buffer properly:

Hi Tomas,

I have just discovered that my fake wget was buggy too, it was
corrupting files... then I followed the first half of your idea - "use
eww" - and I got this, which worked well:

  (require 'eww)
  ;; See: (find-efunction 'eww-download)

  (find-sh0 "rm -v ~/Coetzee99.pdf")
  (setq url "https://tannerlectures.utah.edu/_resources/documents/a-to-z/c/Coetzee99.pdf")
  (url-retrieve url #'eww-download-callback (list url "~/"))
  (find-pdf-page "~/Coetzee99.pdf")

Cheers,
  E. ...

Re: Workshop to save M$ Windows users - help needed

[Tomas Hlavaty]

Hi Eduardo,

On Thu 07 Oct 2021 at 13:32, Eduardo Ochs <eduardoochs@gmail.com> wrote:
> then I followed the first half of your idea - "use eww" - and I got
> this, which worked well:
>
>   (require 'eww)
>   ;; See: (find-efunction 'eww-download)
>
>   (find-sh0 "rm -v ~/Coetzee99.pdf")
>   (setq url "https://tannerlectures.utah.edu/_resources/documents/a-to-z/c/Coetzee99.pdf")
>   (url-retrieve url #'eww-download-callback (list url "/tmp"))
>   (find-pdf-page "~/Coetzee99.pdf")

good you found something that works for you, esp such a simple solution.

Interestingly, I had to remove the "/tmp" argument, otherwise it threw
an error.  This worked for me on Emacs 27.2:

   (setq url "https://tannerlectures.utah.edu/_resources/documents/a-to-z/c/Coetzee99.pdf")
   (url-retrieve url #'eww-download-callback (list url))

It downloads the file in the background.

I wonder, if there is a way to somehow list the running downloads or at
least see the open network buffers?  I do not see them in *Buffer List*
and when downloading with eww, the downloads feel invisible.

Tomas