Emacs source snapshots

Emacs source snapshots

[Grigorios Bouzakis]

Hello, is it possible to provide frequent source snapshots of the
Emacs source tree? For example once a week? The git tree from github [1] [2] is currently > 700 mb, its getting bigger on each commit and pretty fast. I dont know how big the bzr tree currently is. It would be useful for people who dont want to get directly involved with Emacs development, but still want a very recent version of it. For most these people keeping track of commit history doesnt make sense.
AFAIK there is an unofficial repository providing binary packages for Debian and Ubuntu packages [3] plus Ubuntu has emacs-snapshot in their universe repository but package is from 2009 [4]. Hopefully providing frequent snapshots of the sources tree will ease things for everyone as the Emacs sources without commit history are ~100mb. Anyone else finds this a good idea?

[1]: https://github.com/emacsmirror/emacs
[2]: http://www.emacswiki.org/emacs/EmacsFromGit
[3]: http://emacs.naquadah.org
[4]: http://packages.ubuntu.com/search?keywords=emacs-snapshot
-- 
Greg

Re: Emacs source snapshots

[Ted Zlatanov]

On Wed, 04 May 2011 12:39:29 +0300 "Grigorios Bouzakis" <grbzks@xsmail.com> wrote: 

GB> Hello, is it possible to provide frequent source snapshots of the
GB> Emacs source tree? For example once a week? The git tree from github
GB> [1] [2] is currently > 700 mb, its getting bigger on each commit and
GB> pretty fast. I dont know how big the bzr tree currently is. It would
GB> be useful for people who dont want to get directly involved with Emacs
GB> development, but still want a very recent version of it. For most
GB> these people keeping track of commit history doesnt make sense.
GB> AFAIK there is an unofficial repository providing binary packages for
GB> Debian and Ubuntu packages [3] plus Ubuntu has emacs-snapshot in their
GB> universe repository but package is from 2009 [4]. Hopefully providing
GB> frequent snapshots of the sources tree will ease things for everyone
GB> as the Emacs sources without commit history are ~100mb. Anyone else
GB> finds this a good idea?

GB> [1]: https://github.com/emacsmirror/emacs
GB> [2]: http://www.emacswiki.org/emacs/EmacsFromGit
GB> [3]: http://emacs.naquadah.org
GB> [4]: http://packages.ubuntu.com/search?keywords=emacs-snapshot

It's pretty trivial to export the source tree from Bazaar:

# also can use .zip and other formats, see "bzr export --help"
# exports latest revision by default

bzr export snapshot.tgz http://bzr.savannah.gnu.org/r/emacs/trunk/

That could be done nightly and hosted somewhere I guess.  For me it was
a large (700+ MB) download resulting in a 33MB tgz file so it definitely
makes sense for both the server load and the network transfer to host it.

Ted

Re: Emacs source snapshots

[chad]

[-- Attachment #1: Type: text/plain, Size: 384 bytes --]

This seems like a situation where lightweight checkouts might be useful:

http://doc.bazaar.canonical.com/latest/en/user-guide/using_checkouts.html

Hope that helps,
*Chad

On Wed, 04 May 2011 12:39:29 +0300 "Grigorios Bouzakis" <grbzks@xsmail.com> wrote: 

> Hello, is it possible to provide frequent source snapshots of the
> Emacs source tree? For example once a week? 

[-- Attachment #2: Type: text/html, Size: 821 bytes --]

Re: Emacs source snapshots

[Jan Janak]

If you use the git mirror, you may want to consider using the --depth
cmdline option of git-clone. That option can be used to truncate the
commit history to a specific number of commits.

-Jan

On Wed, May 4, 2011 at 5:39 AM, Grigorios Bouzakis <grbzks@xsmail.com> wrote:
> Hello, is it possible to provide frequent source snapshots of the
> Emacs source tree? For example once a week? The git tree from github [1] [2] is currently > 700 mb, its getting bigger on each commit and pretty fast. I dont know how big the bzr tree currently is. It would be useful for people who dont want to get directly involved with Emacs development, but still want a very recent version of it. For most these people keeping track of commit history doesnt make sense.
> AFAIK there is an unofficial repository providing binary packages for Debian and Ubuntu packages [3] plus Ubuntu has emacs-snapshot in their universe repository but package is from 2009 [4]. Hopefully providing frequent snapshots of the sources tree will ease things for everyone as the Emacs sources without commit history are ~100mb. Anyone else finds this a good idea?
>
> [1]: https://github.com/emacsmirror/emacs
> [2]: http://www.emacswiki.org/emacs/EmacsFromGit
> [3]: http://emacs.naquadah.org
> [4]: http://packages.ubuntu.com/search?keywords=emacs-snapshot
> --
> Greg
>
>

Re: Emacs source snapshots

[Grigorios Bouzakis]

Jan Janak <jan@ryngle.com> wrote:
> If you use the git mirror, you may want to consider using the --depth
> cmdline option of git-clone. That option can be used to truncate the
> commit history to a specific number of commits.

I am aware of --depth. But as far as i know thats valid only for the
initial cloning. But cloning the repository a second time to update the
source tree will add  commit history since the first clone was made.
I am not entirely sure about bzr checkout --lightweight but it seems to
work the same way.

----
Greg

Re: Emacs source snapshots

[Leo]

On 2011-05-04 17:39 +0800, Grigorios Bouzakis wrote:
> Hello, is it possible to provide frequent source snapshots of the
> Emacs source tree? For example once a week? The git tree from github
> [1] [2] is currently > 700 mb, its getting bigger on each commit and
> pretty fast. I dont know how big the bzr tree currently is. It would
> be useful for people who dont want to get directly involved with Emacs
> development, but still want a very recent version of it. For most
> these people keeping track of commit history doesnt make sense. AFAIK
> there is an unofficial repository providing binary packages for Debian
> and Ubuntu packages [3] plus Ubuntu has emacs-snapshot in their
> universe repository but package is from 2009 [4]. Hopefully providing
> frequent snapshots of the sources tree will ease things for everyone
> as the Emacs sources without commit history are ~100mb. Anyone else
> finds this a good idea?

You can always get the snapshot from cgit, for example,
http://repo.or.cz/w/emacs.git/snapshot/234c7895577fa0acac1366fe01c1f0b59d6191d5.tar.gz

Leo

Re: Emacs source snapshots

[Stephen J. Turnbull]

Grigorios Bouzakis writes:

 > I am aware of --depth. But as far as i know thats valid only for the
 > initial cloning. But cloning the repository a second time to update the
 > source tree will add  commit history since the first clone was
 > made.

True.  However, if you have enough space to build Emacs comfortably,
you don't need to worry about taking up tons of extra space for a
while (a few hundred or thousand commits).  While the proponents of
the various VCSes will argue about which is most space-efficient, they
are all very space-efficient.

 > I am not entirely sure about bzr checkout --lightweight but it
 > seems to work the same way.

No, it doesn't.  bzr has a plethora of options for connecting
workspaces to repositories.  A lightweight checkout really is a
checkout in the sense you are used to with CVS or Subversion: there is
no repository local to the checkout and *all* version information is
kept in the source repository, except for a simple pointer to the
currect version that is checked out.  This means that all operations
that refer to versioning require communication with the repository.

I have seen reports recently on the bazaar list that use of a
lightweight checkout over the network requires an inordinate amount of
bandwidth, but even so it may be worth while for your usage.
(Lightweight checkouts were originally designed for use with local
shared repositories as a fast way of switching among branches in a
single workspace, and are not optimized to minimize I/O.)

The other main forms of "checkout" in bzr are:
(1) *stacked branches*, which are similar to git limited-depth
    checkouts in that they only store the most recent versions
    locally, but they really are full service branches (they will go
    out and get version information from the parent when necessary).
(2) *bound branches* (aka heavyweight checkouts) which requires a full
    local repository (all versions stored), and keep the local and
    remote repositories in full sync at all times.
(3) *standalone branches* which maintain a full local repository but
    may diverge from the remote repository as appropriate for
    developers.

Re: Emacs source snapshots

[Christian Ohler]

On 4/05/11 5:39, Grigorios Bouzakis wrote:
> Hello, is it possible to provide frequent source snapshots of the
> Emacs source tree?

Try http://hydra.nixos.org/job/gnu/emacs-trunk/tarball/latest .

My understanding is that Hydra continuously runs make-dist for us and 
provides the most recent output at the above URL.

Note that our Hydra setup is still very experimental.

Christian.

Re: Emacs source snapshots

[Andreas Röhler]

Am 05.05.2011 04:58, schrieb Leo:
> On 2011-05-04 17:39 +0800, Grigorios Bouzakis wrote:
>> Hello, is it possible to provide frequent source snapshots of the
>> Emacs source tree? For example once a week? The git tree from github
>> [1] [2] is currently>  700 mb, its getting bigger on each commit and
>> pretty fast. I dont know how big the bzr tree currently is. It would
>> be useful for people who dont want to get directly involved with Emacs
>> development, but still want a very recent version of it. For most
>> these people keeping track of commit history doesnt make sense. AFAIK
>> there is an unofficial repository providing binary packages for Debian
>> and Ubuntu packages [3] plus Ubuntu has emacs-snapshot in their
>> universe repository but package is from 2009 [4]. Hopefully providing
>> frequent snapshots of the sources tree will ease things for everyone
>> as the Emacs sources without commit history are ~100mb. Anyone else
>> finds this a good idea?
>
> You can always get the snapshot from cgit, for example,
> http://repo.or.cz/w/emacs.git/snapshot/234c7895577fa0acac1366fe01c1f0b59d6191d5.tar.gz
>
> Leo
>
>
>

would assume that being snapshot of a specific day.

Let's point at the pure address again:

http://repo.or.cz/w/emacs.git

Re: Emacs source snapshots

[Grigorios Bouzakis]

Christian Ohler <ohler@gnu.org> wrote:
> On 4/05/11 5:39, Grigorios Bouzakis wrote:
>> Hello, is it possible to provide frequent source snapshots of the
>> Emacs source tree?
>
> Try http://hydra.nixos.org/job/gnu/emacs-trunk/tarball/latest .
>
> My understanding is that Hydra continuously runs make-dist for us and 
> provides the most recent output at the above URL.
>
> Note that our Hydra setup is still very experimental.

Indeed, hydra seems to be generating tarballs on each commit that takes
place on the emacs repository. Thats amazing! I couldn't help noticing
though that the files in the tarball that haven't been touched by the
latest commit (or something close to that) are dated Jan 1 1970.
This seems to be the solution  closer to what i would like to be using
so far. The only issue i have with all of the presented ones (thanks for
making them BTW) is that none of them is official. As recent history
with the git repo has taught us, they might not be reliable in the long
run. In fact the git repository at savannah, which i dont know if it can
be considered more official than the others at repo.or.cz and github, is
still stalled since April 1st.
I think i'll be using hydra, at least for now.

Thank you

----
Greg

Re: Emacs source snapshots

[Ted Zlatanov]

On Thu, 05 May 2011 19:39:25 +0200 Andreas Röhler <andreas.roehler@online.de> wrote: 

AR> Let's point at the pure address again:

AR> http://repo.or.cz/w/emacs.git

We just had a long outage of the Git mirror.  Let's not say it is a
viable way to obtain Emacs source snapshots.  It's not.

The Bazaar export command I showed is the best way to get a snapshot of
the trunk without getting the whole repository, using only Bazaar itself
without any external dependencies (you only need Bazaar installed).  

It seems unoptimized currently (so the actual download is large) but at
least it's guaranteed to work and can be hosted somewhere.

Ted

Re: Emacs source snapshots

[chad]

On May 5, 2011, at 12:39 PM, Ted Zlatanov wrote:

> We just had a long outage of the Git mirror.  Let's not say it is a
> viable way to obtain Emacs source snapshots.  It's not.
> 
> The Bazaar export command I showed is the best way to get a snapshot of
> the trunk without getting the whole repository, using only Bazaar itself
> without any external dependencies (you only need Bazaar installed).  
> 
> It seems unoptimized currently (so the actual download is large) but at
> least it's guaranteed to work and can be hosted somewhere.
> 
> Ted

The lightweight checkout method I mentioned seems to work without
downloading so much, and gives a working albeit slow history, using 
only bzr.

I just tried it now:

	; bzr checkout --lightweight http://bzr.savannah.gnu.org/r/emacs/trunk/
	; du -sk trunk/
	120064	trunk/

for comparison, my normal checkout with history:

	; du -sk trunk .bzr
	120256	trunk
	388052	.bzr
 
For annoying reasons, I didn't capture exactly how much bzr reported 
as downloading, but it seemed to be about 200m.
 
Hope that helps,
*Chad

Re: Emacs source snapshots

[Ted Zlatanov]

On Thu, 5 May 2011 16:29:02 -0700 chad <yandros@MIT.EDU> wrote: 

c> On May 5, 2011, at 12:39 PM, Ted Zlatanov wrote:

>> The Bazaar export command I showed is the best way to get a snapshot of
>> the trunk without getting the whole repository, using only Bazaar itself
>> without any external dependencies (you only need Bazaar installed).  
>> 
>> It seems unoptimized currently (so the actual download is large) but at
>> least it's guaranteed to work and can be hosted somewhere.

c> The lightweight checkout method I mentioned seems to work without
c> downloading so much, and gives a working albeit slow history, using 
c> only bzr.

Yes, that seems good too.  As long as it works without relying on the
Git mirror...

Ted

Re: Emacs source snapshots

[Eli Zaretskii]

> From: chad <yandros@MIT.EDU>
> Date: Thu, 5 May 2011 16:29:02 -0700
> Cc: emacs-devel@gnu.org
> 
> For annoying reasons, I didn't capture exactly how much bzr reported 
> as downloading, but it seemed to be about 200m.

You can find that info recorded in your .bzr.log file.