* [PATCH] Add an 'optimiseStore' RPC
@ 2014-08-31 13:44 Ludovic Courtès
2014-09-01 19:53 ` Eelco Dolstra
0 siblings, 1 reply; 4+ messages in thread
From: Ludovic Courtès @ 2014-08-31 13:44 UTC (permalink / raw)
To: Eelco Dolstra; +Cc: guix-devel, nix-dev
[-- Attachment #1: Type: text/plain, Size: 167 bytes --]
Hi Eelco,
The patch below adds an ‘optimiseStore’ RPC, and thus adds a mandatory
‘optimiseStore’ method in ‘StoreAPI’.
OK to commit?
Ludo’.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 4751 bytes --]
diff --git a/src/libstore/local-store.hh b/src/libstore/local-store.hh
index 09639e7..4404ffc 100644
--- a/src/libstore/local-store.hh
+++ b/src/libstore/local-store.hh
@@ -168,6 +168,9 @@ public:
files with the same contents. */
void optimiseStore(OptimiseStats & stats);
+ /* Generic variant of the above method. */
+ void optimiseStore();
+
/* Optimise a single store path. */
void optimisePath(const Path & path);
diff --git a/src/libstore/optimise-store.cc b/src/libstore/optimise-store.cc
index d833f3a..68acb6e 100644
--- a/src/libstore/optimise-store.cc
+++ b/src/libstore/optimise-store.cc
@@ -169,6 +169,24 @@ void LocalStore::optimiseStore(OptimiseStats & stats)
}
}
+static string showBytes(unsigned long long bytes)
+{
+ return (format("%.2f MiB") % (bytes / (1024.0 * 1024.0))).str();
+}
+
+void LocalStore::optimiseStore()
+{
+ OptimiseStats stats;
+
+ optimiseStore(stats);
+
+ printMsg(lvlError,
+ format("%1% freed by hard-linking %2% files; there are %3% files with equal contents out of %4% files in total")
+ % showBytes(stats.bytesFreed)
+ % stats.filesLinked
+ % stats.sameContents
+ % stats.totalFiles);
+}
void LocalStore::optimisePath(const Path & path)
{
diff --git a/src/libstore/remote-store.cc b/src/libstore/remote-store.cc
index 4619206..8e1bfdb 100644
--- a/src/libstore/remote-store.cc
+++ b/src/libstore/remote-store.cc
@@ -564,6 +564,13 @@ void RemoteStore::clearFailedPaths(const PathSet & paths)
readInt(from);
}
+void RemoteStore::optimiseStore()
+{
+ openConnection();
+ writeInt(wopOptimiseStore, to);
+ processStderr();
+ readInt(from);
+}
void RemoteStore::processStderr(Sink * sink, Source * source)
{
diff --git a/src/libstore/remote-store.hh b/src/libstore/remote-store.hh
index 04b60fc..b9b7f7c 100644
--- a/src/libstore/remote-store.hh
+++ b/src/libstore/remote-store.hh
@@ -82,7 +82,9 @@ public:
PathSet queryFailedPaths();
void clearFailedPaths(const PathSet & paths);
-
+
+ void optimiseStore();
+
private:
AutoCloseFD fdSocket;
FdSink to;
diff --git a/src/libstore/store-api.hh b/src/libstore/store-api.hh
index b635fee..3109f10 100644
--- a/src/libstore/store-api.hh
+++ b/src/libstore/store-api.hh
@@ -250,6 +250,10 @@ public:
`nix-store --register-validity'. */
string makeValidityRegistration(const PathSet & paths,
bool showDerivers, bool showHash);
+
+ /* Optimise the disk space usage of the Nix store by hard-linking files
+ with the same contents. */
+ virtual void optimiseStore() = 0;
};
diff --git a/src/libstore/worker-protocol.hh b/src/libstore/worker-protocol.hh
index 9317f89..9a3d83a 100644
--- a/src/libstore/worker-protocol.hh
+++ b/src/libstore/worker-protocol.hh
@@ -43,6 +43,7 @@ typedef enum {
wopQueryValidPaths = 31,
wopQuerySubstitutablePaths = 32,
wopQueryValidDerivers = 33,
+ wopOptimiseStore = 34
} WorkerOp;
diff --git a/src/nix-daemon/nix-daemon.cc b/src/nix-daemon/nix-daemon.cc
index 8814fe3..413db20 100644
--- a/src/nix-daemon/nix-daemon.cc
+++ b/src/nix-daemon/nix-daemon.cc
@@ -650,6 +650,13 @@ static void performOp(bool trusted, unsigned int clientVersion,
break;
}
+ case wopOptimiseStore:
+ startWork();
+ store->optimiseStore();
+ stopWork();
+ writeInt(1, to);
+ break;
+
default:
throw Error(format("invalid operation %1%") % op);
}
diff --git a/src/nix-store/nix-store.cc b/src/nix-store/nix-store.cc
index 341a4f6..e13bfe9 100644
--- a/src/nix-store/nix-store.cc
+++ b/src/nix-store/nix-store.cc
@@ -798,17 +798,6 @@ static void opRepairPath(Strings opFlags, Strings opArgs)
}
-static void showOptimiseStats(OptimiseStats & stats)
-{
- printMsg(lvlError,
- format("%1% freed by hard-linking %2% files; there are %3% files with equal contents out of %4% files in total")
- % showBytes(stats.bytesFreed)
- % stats.filesLinked
- % stats.sameContents
- % stats.totalFiles);
-}
-
-
/* Optimise the disk space usage of the Nix store by hard-linking
files with the same contents. */
static void opOptimise(Strings opFlags, Strings opArgs)
@@ -816,17 +805,9 @@ static void opOptimise(Strings opFlags, Strings opArgs)
if (!opArgs.empty() || !opFlags.empty())
throw UsageError("no arguments expected");
- OptimiseStats stats;
- try {
- ensureLocalStore().optimiseStore(stats);
- } catch (...) {
- showOptimiseStats(stats);
- throw;
- }
- showOptimiseStats(stats);
+ store->optimiseStore();
}
-
static void opQueryFailedPaths(Strings opFlags, Strings opArgs)
{
if (!opArgs.empty() || !opFlags.empty())
[-- Attachment #3: Type: text/plain, Size: 149 bytes --]
_______________________________________________
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] Add an 'optimiseStore' RPC
2014-08-31 13:44 [PATCH] Add an 'optimiseStore' RPC Ludovic Courtès
@ 2014-09-01 19:53 ` Eelco Dolstra
2014-09-01 21:18 ` Ludovic Courtès
0 siblings, 1 reply; 4+ messages in thread
From: Eelco Dolstra @ 2014-09-01 19:53 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: guix-devel, nix-dev
Hi,
On 31/08/14 15:44, Ludovic Courtès wrote:
> The patch below adds an ‘optimiseStore’ RPC, and thus adds a mandatory
> ‘optimiseStore’ method in ‘StoreAPI’.
Looks fine, but it doesn't apply. Can you rebase it on nix master?
--
Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/
_______________________________________________
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] Add an 'optimiseStore' RPC
2014-09-01 19:53 ` Eelco Dolstra
@ 2014-09-01 21:18 ` Ludovic Courtès
2014-09-01 21:55 ` Eelco Dolstra
0 siblings, 1 reply; 4+ messages in thread
From: Ludovic Courtès @ 2014-09-01 21:18 UTC (permalink / raw)
To: Eelco Dolstra; +Cc: guix-devel, nix-dev
[-- Attachment #1: Type: text/plain, Size: 177 bytes --]
Eelco Dolstra <eelco.dolstra@logicblox.com> skribis:
> Looks fine, but it doesn't apply. Can you rebase it on nix master?
Oops, sure. Here it is.
Thanks,
Ludo’.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: the patch --]
[-- Type: text/x-patch, Size: 5178 bytes --]
From 88ebc46a1216165a015a744e53319f666eb1b1cb Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
Date: Mon, 1 Sep 2014 22:21:42 +0200
Subject: [PATCH] Add an 'optimiseStore' remote procedure call.
---
src/libstore/local-store.hh | 3 +++
src/libstore/optimise-store.cc | 16 ++++++++++++++++
src/libstore/remote-store.cc | 7 +++++++
src/libstore/remote-store.hh | 4 +++-
src/libstore/store-api.hh | 4 ++++
src/libstore/worker-protocol.hh | 1 +
src/nix-daemon/nix-daemon.cc | 7 +++++++
src/nix-store/nix-store.cc | 20 +-------------------
8 files changed, 42 insertions(+), 20 deletions(-)
diff --git a/src/libstore/local-store.hh b/src/libstore/local-store.hh
index e58e656..dccdba5 100644
--- a/src/libstore/local-store.hh
+++ b/src/libstore/local-store.hh
@@ -167,6 +167,9 @@ public:
files with the same contents. */
void optimiseStore(OptimiseStats & stats);
+ /* Generic variant of the above method. */
+ void optimiseStore();
+
/* Optimise a single store path. */
void optimisePath(const Path & path);
diff --git a/src/libstore/optimise-store.cc b/src/libstore/optimise-store.cc
index 208d968..dd18d66 100644
--- a/src/libstore/optimise-store.cc
+++ b/src/libstore/optimise-store.cc
@@ -225,6 +225,22 @@ void LocalStore::optimiseStore(OptimiseStats & stats)
}
}
+static string showBytes(unsigned long long bytes)
+{
+ return (format("%.2f MiB") % (bytes / (1024.0 * 1024.0))).str();
+}
+
+void LocalStore::optimiseStore()
+{
+ OptimiseStats stats;
+
+ optimiseStore(stats);
+
+ printMsg(lvlError,
+ format("%1% freed by hard-linking %2% files")
+ % showBytes(stats.bytesFreed)
+ % stats.filesLinked);
+}
void LocalStore::optimisePath(const Path & path)
{
diff --git a/src/libstore/remote-store.cc b/src/libstore/remote-store.cc
index 8ef6737..a0e9f22 100644
--- a/src/libstore/remote-store.cc
+++ b/src/libstore/remote-store.cc
@@ -579,6 +579,13 @@ void RemoteStore::clearFailedPaths(const PathSet & paths)
readInt(from);
}
+void RemoteStore::optimiseStore()
+{
+ openConnection();
+ writeInt(wopOptimiseStore, to);
+ processStderr();
+ readInt(from);
+}
void RemoteStore::processStderr(Sink * sink, Source * source)
{
diff --git a/src/libstore/remote-store.hh b/src/libstore/remote-store.hh
index b010147..98774c1 100644
--- a/src/libstore/remote-store.hh
+++ b/src/libstore/remote-store.hh
@@ -82,7 +82,9 @@ public:
PathSet queryFailedPaths();
void clearFailedPaths(const PathSet & paths);
-
+
+ void optimiseStore();
+
private:
AutoCloseFD fdSocket;
FdSink to;
diff --git a/src/libstore/store-api.hh b/src/libstore/store-api.hh
index b635fee..3109f10 100644
--- a/src/libstore/store-api.hh
+++ b/src/libstore/store-api.hh
@@ -250,6 +250,10 @@ public:
`nix-store --register-validity'. */
string makeValidityRegistration(const PathSet & paths,
bool showDerivers, bool showHash);
+
+ /* Optimise the disk space usage of the Nix store by hard-linking files
+ with the same contents. */
+ virtual void optimiseStore() = 0;
};
diff --git a/src/libstore/worker-protocol.hh b/src/libstore/worker-protocol.hh
index c7d3a72..4b040b7 100644
--- a/src/libstore/worker-protocol.hh
+++ b/src/libstore/worker-protocol.hh
@@ -42,6 +42,7 @@ typedef enum {
wopQueryValidPaths = 31,
wopQuerySubstitutablePaths = 32,
wopQueryValidDerivers = 33,
+ wopOptimiseStore = 34
} WorkerOp;
diff --git a/src/nix-daemon/nix-daemon.cc b/src/nix-daemon/nix-daemon.cc
index ced356c..0f3235d 100644
--- a/src/nix-daemon/nix-daemon.cc
+++ b/src/nix-daemon/nix-daemon.cc
@@ -508,6 +508,13 @@ static void performOp(bool trusted, unsigned int clientVersion,
break;
}
+ case wopOptimiseStore:
+ startWork();
+ store->optimiseStore();
+ stopWork();
+ writeInt(1, to);
+ break;
+
default:
throw Error(format("invalid operation %1%") % op);
}
diff --git a/src/nix-store/nix-store.cc b/src/nix-store/nix-store.cc
index 8c37448..51839fa 100644
--- a/src/nix-store/nix-store.cc
+++ b/src/nix-store/nix-store.cc
@@ -823,16 +823,6 @@ static void opRepairPath(Strings opFlags, Strings opArgs)
}
}
-
-static void showOptimiseStats(OptimiseStats & stats)
-{
- printMsg(lvlError,
- format("%1% freed by hard-linking %2% files")
- % showBytes(stats.bytesFreed)
- % stats.filesLinked);
-}
-
-
/* Optimise the disk space usage of the Nix store by hard-linking
files with the same contents. */
static void opOptimise(Strings opFlags, Strings opArgs)
@@ -840,17 +830,9 @@ static void opOptimise(Strings opFlags, Strings opArgs)
if (!opArgs.empty() || !opFlags.empty())
throw UsageError("no arguments expected");
- OptimiseStats stats;
- try {
- ensureLocalStore().optimiseStore(stats);
- } catch (...) {
- showOptimiseStats(stats);
- throw;
- }
- showOptimiseStats(stats);
+ store->optimiseStore();
}
-
static void opQueryFailedPaths(Strings opFlags, Strings opArgs)
{
if (!opArgs.empty() || !opFlags.empty())
--
1.8.4
[-- Attachment #3: Type: text/plain, Size: 149 bytes --]
_______________________________________________
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] Add an 'optimiseStore' RPC
2014-09-01 21:18 ` Ludovic Courtès
@ 2014-09-01 21:55 ` Eelco Dolstra
0 siblings, 0 replies; 4+ messages in thread
From: Eelco Dolstra @ 2014-09-01 21:55 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: guix-devel, nix-dev
Hi,
On 01/09/14 23:18, Ludovic Courtès wrote:
> Eelco Dolstra <eelco.dolstra@logicblox.com> skribis:
>
>> Looks fine, but it doesn't apply. Can you rebase it on nix master?
>
> Oops, sure. Here it is.
Thanks, applied!
--
Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/
_______________________________________________
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2014-09-01 21:55 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-08-31 13:44 [PATCH] Add an 'optimiseStore' RPC Ludovic Courtès
2014-09-01 19:53 ` Eelco Dolstra
2014-09-01 21:18 ` Ludovic Courtès
2014-09-01 21:55 ` Eelco Dolstra
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).