my CDN measure_get

my CDN measure_get

[zimoun]

Dear,

I do not know if it is useful because it appears to me that providing
CDN service is now on the pipes.

Well, even if I understand all the concerns mitigating the use of
external CDN, I am in favor of this use.
Mainly, because I use guix in scientific context. And it is hard to
explain to my (hypothetical) collaborators that they have to wait.

The figures below are from my laptop connected by Wifi in my
university and then from the cluster head node. I am in Paris.
Note that the default curl version on the cluster is old.


I am not sure to know how to interpret but I see that the CDN seems much faster.


Last, what should be the cost to enable CDN? Any evaluation?


Hope that helps.

All the best,
simon

--


laptop$ measure_get
https://berlin.guixsd.org/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
2>/dev/null
url_effective: https://berlin.guixsd.org/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
http_code: 200
num_connects: 1
num_redirects: 0
remote_ip: 141.80.181.40
remote_port: 443
size_download: 69899433 B
speed_download: 2088170.000 B/s
time_appconnect: 0.123089 s
time_connect: 0.042923 s
time_namelookup: 0.004140 s
time_pretransfer: 0.123141 s
time_redirect: 0.000000 s
time_starttransfer: 0.199722 s
time_total: 33.474630 s


laptop$ measure_get
https://berlin-mirror.marusich.info/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
2>/dev/null
url_effective: https://berlin-mirror.marusich.info/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
http_code: 200
num_connects: 1
num_redirects: 0
remote_ip: 143.204.229.90
remote_port: 443
size_download: 69899433 B
speed_download: 10793612.000 B/s
time_appconnect: 0.024161 s
time_connect: 0.010927 s
time_namelookup: 0.007414 s
time_pretransfer: 0.024253 s
time_redirect: 0.000000 s
time_starttransfer: 0.029704 s
time_total: 6.476473 s


cluster$ measure_get
https://berlin.guixsd.org/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
2>/dev/null
url_effective: https://berlin.guixsd.org/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
http_code: 200
num_connects: 1
num_redirects: 0
remote_ip:
remote_port:
size_download: 69899433 B
speed_download: 2863050,000 B/s
time_appconnect: 0,272 s
time_connect: 0,041 s
time_namelookup: 0,004 s
time_pretransfer: 0,272 s
time_redirect: 0,000 s
time_starttransfer: 0,346 s
time_total: 24,414 s


cluster$ measure_get
https://berlin-mirror.marusich.info/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
2>/dev/null
url_effective: https://berlin-mirror.marusich.info/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
http_code: 200
num_connects: 1
num_redirects: 0
remote_ip:
remote_port:
size_download: 69899433 B
speed_download: 43271174,000 B/s
time_appconnect: 0,179 s
time_connect: 0,006 s
time_namelookup: 0,004 s
time_pretransfer: 0,179 s
time_redirect: 0,000 s
time_starttransfer: 0,182 s
time_total: 1,615 s

Re: my CDN measure_get

[swedebugia]

On 2018-12-18 21:23, zimoun wrote:
> Dear,
> 
> I do not know if it is useful because it appears to me that providing
> CDN service is now on the pipes.
> 
> Well, even if I understand all the concerns mitigating the use of
> external CDN, I am in favor of this use.
> Mainly, because I use guix in scientific context. And it is hard to
> explain to my (hypothetical) collaborators that they have to wait.

LOL. Waiting is a challenge.

snip
> 
> 
> laptop$ measure_get
> https://berlin.guixsd.org/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
> 2>/dev/null
> url_effective: https://berlin.guixsd.org/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
> http_code: 200
> num_connects: 1
> num_redirects: 0
> remote_ip: 141.80.181.40
> remote_port: 443
> size_download: 69899433 B
> speed_download: 2088170.000 B/s
> time_appconnect: 0.123089 s
> time_connect: 0.042923 s
> time_namelookup: 0.004140 s
> time_pretransfer: 0.123141 s
> time_redirect: 0.000000 s
> time_starttransfer: 0.199722 s
> time_total: 33.474630 s
> 
> 
> laptop$ measure_get
> https://berlin-mirror.marusich.info/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
> 2>/dev/null
> url_effective: https://berlin-mirror.marusich.info/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
> http_code: 200
> num_connects: 1
> num_redirects: 0
> remote_ip: 143.204.229.90
> remote_port: 443
> size_download: 69899433 B
> speed_download: 10793612.000 B/s
> time_appconnect: 0.024161 s
> time_connect: 0.010927 s
> time_namelookup: 0.007414 s
> time_pretransfer: 0.024253 s
> time_redirect: 0.000000 s
> time_starttransfer: 0.029704 s
> time_total: 6.476473 s
> 
> 
> cluster$ measure_get
> https://berlin.guixsd.org/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
> 2>/dev/null
> url_effective: https://berlin.guixsd.org/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
> http_code: 200
> num_connects: 1
> num_redirects: 0
> remote_ip:
> remote_port:
> size_download: 69899433 B
> speed_download: 2863050,000 B/s
> time_appconnect: 0,272 s
> time_connect: 0,041 s
> time_namelookup: 0,004 s
> time_pretransfer: 0,272 s
> time_redirect: 0,000 s
> time_starttransfer: 0,346 s
> time_total: 24,414 s
> 
> 
> cluster$ measure_get
> https://berlin-mirror.marusich.info/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
> 2>/dev/null
> url_effective: https://berlin-mirror.marusich.info/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
> http_code: 200
> num_connects: 1
> num_redirects: 0
> remote_ip:
> remote_port:
> size_download: 69899433 B
> speed_download: 43271174,000 B/s
> time_appconnect: 0,179 s
> time_connect: 0,006 s
> time_namelookup: 0,004 s
> time_pretransfer: 0,179 s
> time_redirect: 0,000 s
> time_starttransfer: 0,182 s
> time_total: 1,615 s
> 

Did you run the CDNs twice at the first try?

Interesting measures. It seems berlin max out at about 3MB/s. The CDN 
is of course much faster because it is probably closer to Paris.

Ricardo, what is the uplink of berlin? 1Gbps?

-- 
Cheers Swedebugia

Re: my CDN measure_get

[zimoun]

> Did you run the CDNs twice at the first try?

Yes, I launched the command several times to see differences. There
are not so much. I reported my last experiment.


> Interesting measures. It seems berlin max out at about 3MB/s. The CDN
> is of course much faster because it is probably closer to Paris.

Maybe it is even in Paris :-)


All the best,
simon

Re: my CDN measure_get

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 3741 bytes --]

Hi Simon,

Thank you for sharing the data with us!  It seems that for you, in
Paris, CloudFront represents a very nice performance improvement.

zimoun <zimon.toutoune@gmail.com> writes:

> laptop$ measure_get
> https://berlin.guixsd.org/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
> 2>/dev/null
> url_effective: https://berlin.guixsd.org/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
> http_code: 200
> num_connects: 1
> num_redirects: 0
> remote_ip: 141.80.181.40
> remote_port: 443
> size_download: 69899433 B
> speed_download: 2088170.000 B/s
> time_appconnect: 0.123089 s
> time_connect: 0.042923 s
> time_namelookup: 0.004140 s
> time_pretransfer: 0.123141 s
> time_redirect: 0.000000 s
> time_starttransfer: 0.199722 s
> time_total: 33.474630 s

For this request, berlin.guixsd.org throughput is 17 Mbps, and latency
is 39 ms after DNS name resolution.

> laptop$ measure_get
> https://berlin-mirror.marusich.info/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
> 2>/dev/null
> url_effective: https://berlin-mirror.marusich.info/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
> http_code: 200
> num_connects: 1
> num_redirects: 0
> remote_ip: 143.204.229.90
> remote_port: 443
> size_download: 69899433 B
> speed_download: 10793612.000 B/s
> time_appconnect: 0.024161 s
> time_connect: 0.010927 s
> time_namelookup: 0.007414 s
> time_pretransfer: 0.024253 s
> time_redirect: 0.000000 s
> time_starttransfer: 0.029704 s
> time_total: 6.476473 s

For this request, berlin-mirror.marusich.info throughput is 86 Mbps, and
latency is 3.5 ms after DNS name resolution.  The throughput is about
400% greater than going directly to berlin.guixsd.org, and the latency
is about 91% less than going directly to berlin.guixsd.org.

> cluster$ measure_get
> https://berlin.guixsd.org/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
> 2>/dev/null
> url_effective: https://berlin.guixsd.org/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
> http_code: 200
> num_connects: 1
> num_redirects: 0
> remote_ip:
> remote_port:
> size_download: 69899433 B
> speed_download: 2863050,000 B/s
> time_appconnect: 0,272 s
> time_connect: 0,041 s
> time_namelookup: 0,004 s
> time_pretransfer: 0,272 s
> time_redirect: 0,000 s
> time_starttransfer: 0,346 s
> time_total: 24,414 s

For this request, berlin.guixsd.org throughput is 23 Mbps, and latency
is 37 ms after DNS name resolution.  Compared to the previous direct
request to berlin.guixsd.org, the throughput is 35% greater, and the
latency is 5.1% less.

> cluster$ measure_get
> https://berlin-mirror.marusich.info/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
> 2>/dev/null
> url_effective: https://berlin-mirror.marusich.info/nar/gzip/1bq783rbkzv9z9zdhivbvfzhsz2s5yac-linux-libre-4.19
> http_code: 200
> num_connects: 1
> num_redirects: 0
> remote_ip:
> remote_port:
> size_download: 69899433 B
> speed_download: 43271174,000 B/s
> time_appconnect: 0,179 s
> time_connect: 0,006 s
> time_namelookup: 0,004 s
> time_pretransfer: 0,179 s
> time_redirect: 0,000 s
> time_starttransfer: 0,182 s
> time_total: 1,615 s

For this request, berlin-mirror.marusich.info throughput is 346 Mbps,
and latency is 2 ms after DNS name resolution.  Compared to the direct
request to berlin.guixsd.org, this request's throughput is 1400%
greater, and its latency is 95% lower.  I suppose your cluster must be
close to a CloudFront edge location.

So yes, I think this makes it quite clear that for you, CloudFront
represents a very nice performance improvement, both on your laptop and
especially in your cluster.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]