From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Daniel Llorens Newsgroups: gmane.lisp.guile.devel Subject: attempt at array cleanup Date: Wed, 1 May 2013 01:10:17 +0200 Message-ID: <918054D0-5A95-483E-B74A-675230709646@bluewin.ch> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: multipart/mixed; boundary=Apple-Mail-87-361677366 X-Trace: ger.gmane.org 1367363432 9645 80.91.229.3 (30 Apr 2013 23:10:32 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 30 Apr 2013 23:10:32 +0000 (UTC) To: guile-devel Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Wed May 01 01:10:32 2013 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1UXJgt-0001Iy-IW for guile-devel@m.gmane.org; Wed, 01 May 2013 01:10:31 +0200 Original-Received: from localhost ([::1]:48709 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UXJgs-0000j3-VH for guile-devel@m.gmane.org; Tue, 30 Apr 2013 19:10:30 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:55870) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UXJgm-0000iw-FG for guile-devel@gnu.org; Tue, 30 Apr 2013 19:10:26 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UXJgk-0004LU-NP for guile-devel@gnu.org; Tue, 30 Apr 2013 19:10:24 -0400 Original-Received: from zhbdzmsp-smta17.bluewin.ch ([195.186.99.133]:48507) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UXJgk-0004LB-Al for guile-devel@gnu.org; Tue, 30 Apr 2013 19:10:22 -0400 Original-Received: from [195.186.227.131] ([195.186.227.131:53455] helo=zhhdzmsp-smta14.bluewin.ch) by zhbdzmsp-smta17.bluewin.ch (envelope-from ) (ecelerity 2.2.3.47 r(39824M)) with ESMTP id 40/C2-11581-A5F40815; Tue, 30 Apr 2013 23:10:19 +0000 Original-Received: from [10.0.1.10] (83.78.57.14) by zhhdzmsp-smta14.bluewin.ch (8.5.142) (authenticated as dll@bluewin.ch) id 511CD83E06B95F00 for guile-devel@gnu.org; Tue, 30 Apr 2013 23:10:18 +0000 X-Mailer: Apple Mail (2.1085) X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 195.186.99.133 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:16323 Archived-At: --Apple-Mail-87-361677366 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hello, a few weeks ago, a few bugs and some peculiar performance behavior = prompted me to start looking at the array code in Guile. I have put what I've done on branch ra0 here: = https://gitorious.org/guile-stable-2-0-array-cleanup/guile-stable-2-0-arra= y-cleanup/commits/ra0 This is ~50 commits on top of stable-2.0. I'm doing a few different things in these patches. ----------------------------- A. The following bugs are fixed, compared with stable-2.0. A1. strict flag for array-contents doesn't work properly. In stable-2.0: (define (every-two x) (make-shared-array x (lambda (i) (list (* i 2))) = 2)) (define a (every-two #f64(1 2 3 4))) (array-contents a #t) =3D> #f64(1.0 3.0) This should be #f. A2. transpose-array doesn't work on 0-rank arrays. In stable-2.0: (transpose-array #0(99)) =3D> bad argument list This should be #0(99). A3. bad access with array ops when the array is empty but the last axis = is not. In stable-2.0: (array-index-map! (make-typed-array 'f64 0 0 2) (const 0)) =3D> value out of range (array-index-map! (make-typed-array #t 0 0 2) (const 0)) =3D> segfault/nop, according to luck (this is a second bug, friends with = the first). Both should be nops. ----------------------------- B. Implementation changes. There are (loosely speaking) 4 types in Guile that can be used as array = roots: bytevectors, bitvectors, strings and vectors. However, of these, = bitvectors and vectors also accept array arguments, which results in a = lot of back and forth between the array procedures and the bitvector and = vector procedures. Following the 1st column of = http://lists.gnu.org/archive/html/guile-devel/2013-04/msg00176.html, = vector-ref/set!/length and uniform-vector-ref/set!/length are no longer = type generic (they don't accept objects with the Guile array tag). This removes some logical errors like (vector-ref #@1(1 2 3) 1) =3D> 2. = There was some discussion in that thread, and it would be good to have = the matter settled either way. ----------------------------- C. Performance. I don't store the array in the array handle, but the root instead. This = removes a level of indirection where array-ref/set! had to go first = through the array 'implementation' to get to the root's -ref function. = Now it goes there directly. I've also fixed a number of cases where the = array handle procedures were used even on an object that was known to = have the array tag, some redundant checks, etc. I have rewritten scm_ramapc (used by array-map!, array-for-each, = array-copy!). The previous version attempt full unrolling, and if it = failed, it only unrolled the last axis. The new version is simpler and = may unroll any number of axes from the end, so e.g. the last one, the = last two, etc. This gives a big speed up in some cases. Obvious next = steps would be 1) to grade the axes by stride, to allow unrolling from = other than the last axis; and 2) to do all type dispatch outside the = loops. This would bloat array-map.c though. I have been tracking a small benchmark, which is attached at the end. ----------------------------- D. Test cases. I've added test cases for the array functions, beyond the bugs above. = These are all in test-suite/tests/ramap.test. There're some other tests. = All the tests in stable-2.0 pass unchanged. ----------------------------- E. Unfixed bugs and new. In stable-2.0, this fails at the REPL: scheme@(guile-user)> #1a@1(#\a #\b) While compiling expression: ERROR: Wrong type (expecting uniform array): #1a@1(#\a #\b) The error is in uniform-array->bytevector: CHAR arrays don't have an = 'elements' pointer. In the ra0 branch, that still fails, and this also fails: scheme@(guile-user)> #1b@1(#t #t) While compiling expression: ERROR: In procedure uniform-array->bytevector: Wrong type (expecting = uniform contiguous array): #1b@1(#t #t) In both stable-2.0 and ra0, (bitvector? (make-typed-array 'b 10)) =3D> #f This is different from what happens with other types, where = make-typed-array always tries to return the root (not sure I agree with = this, but at least is semi-consistent). Maybe the compiler should use = the actual array procedures here instead of uniform-array->bytevector, = since those are safe wrt 'arrayness'. Regards, Daniel ----------------------------- The benchmark. I'd listen to recommendations on proper benchmarking = setups. I hope it's ok to post the figure here; the abscissas are = commits between stable-2.0 and ra0. --%------- (define-syntax time (syntax-rules () ((_ e0 ...) (let ((start (tms:clock (times)))) e0 ... (exact->inexact (/ (- (tms:clock (times)) start) internal-time-units-per-second)))))) (define-syntax repeat (syntax-rules () ((_ n e0 ...) (do ((i 0 (+ i 1))) ((=3D i n)) e0 ...)))) (define (vector-index-map-1! q f) (let ((n (vector-length q))) (do ((i 0 (+ i 1))) ((=3D i n)) (vector-set! q i (f i))))) (define (array-index-map-1! q f) (let ((n (array-length q))) (do ((i 0 (+ i 1))) ((=3D i n)) (array-set! q (f i) i)))) (define (array-index-map-2! q f) (let* ((n (array-dimensions q)) (n0 (car n)) (n1 (cadr n))) (do ((i 0 (+ i 1))) ((=3D i n0)) (do ((j 0 (+ j 1))) ((=3D j n1)) (array-set! q (f i j) i j))))) (define a (make-array 0. 100000 10)) (define b (make-array 0. 100000 10)) (define c (make-array *unspecified* 100000 10)) (define d (transpose-array (make-array *unspecified* 10 100000) 1 0)) (define aa (make-typed-array 'a *unspecified* 1000000 10)) (define s "1234567890") (define t (make-shared-array s (lambda (i j) (list j)) 1000000 10)) (define x (make-typed-array 'a *unspecified* 1000000 10)) (define y (make-typed-array 'a *unspecified* 1000000 10)) (define i (make-typed-array #t *unspecified* 1000000)) (define j (make-typed-array #t *unspecified* 1000000 10)) (define k (make-vector 1000000 *unspecified*)) (define u #2((1 2 3) (4 5 6) (7 8 9))) (define v (make-shared-array u (lambda (i j k) (list j k)) 1000000 3 3)) (define w (make-array 0 1000000 3 3)) (define o (make-shared-array #0(99) (lambda x '()) 2000 2000 10)) (define p (make-array 0 2000 2000 10)) (define a0 (make-array 1. 1000000 100)) (define b0 (make-array *unspecified* 1000000 100)) (define c0 (transpose-array (make-array *unspecified* 100 1000000) 1 0)) (define a1 (make-array 1. 100 1000000)) (define b1 (make-array *unspecified* 100 1000000)) (define c1 (transpose-array (make-array *unspecified* 1000000 100) 1 0)) (define out (string-append (cadr (program-arguments)) ".bench")) (define (wout name t) (format #t "~a ~a\n" name t)) (with-output-to-file out (lambda () (wout "map0" (time (repeat 20 (array-map! c (const 0.))))) (wout "map1a" (time (repeat 20 (array-map! c (lambda (x) x) a)))) (wout "map1b" (time (repeat 20 (array-map! c (lambda (x) x) d)))) (wout "map2" (time (repeat 5 (array-map! c (lambda (a b) (+ a b)) a = b)))) (wout "fill-char1" (time (repeat 3 (array-fill! aa #\v)))) (wout "cp-char2a" (time (repeat 3 (array-copy! t x)))) (wout "cp-char2b" (time (repeat 3 (array-copy! x y)))) (wout "aim1" (time (repeat 20 (array-index-map! i (lambda (i) i))))) (wout "aim2" (time (repeat 2 (array-index-map! j (lambda (i j) = i))))) (wout "vset1" (time (repeat 30 (vector-index-map-1! k (lambda (i) = i))))) (wout "aset1" (time (repeat 10 (array-index-map-1! i (lambda (i) = i))))) (wout "aset2" (time (repeat 1 (array-index-map-2! j (lambda (i j) = i))))) (wout "cp3a" (time (repeat 30 (array-copy! v w)))) (wout "cp3b" (time (repeat 30 (array-copy! o p)))) (wout "cp2a" (time (repeat 10 (array-copy! a0 b0)))) (wout "cp2b" (time (repeat 3 (array-copy! a0 c0)))) (wout "cp2c" (time (repeat 10 (array-copy! a1 b1)))) (wout "cp2d" (time (repeat 1 (array-copy! a1 c1)))))) --Apple-Mail-87-361677366 Content-Disposition: inline; filename=bench.pdf Content-Type: application/pdf; x-unix-mode=0644; name="bench.pdf" Content-Transfer-Encoding: base64 JVBERi0xLjQKJcfsj6IKNSAwIG9iago8PC9MZW5ndGggNiAwIFIvRmlsdGVyIC9GbGF0ZURlY29k ZT4+CnN0cmVhbQp4nKVcTbMlt23dv19xl3aq/MLvJrepSmWVhW3tZC9GY0lxaqYij1zx3w+JcwCC 9743ka80VRqiQRDs7sNDAOw7f7uF13gL6w///vj55V//cN1+/PmlvV5XK9ftHy8xvJZ8+88X6Xf7 w3+8xNTTrbZ0i7GmcLtai7cv37/8kT2+/Pjyt2kkbf718fPt376ZI/db7q85lNs3P7zAYZyDXK+1 plt/7XPMevvm88tvwm+/+e+Xf//m5ffw1VPKt89shzZe6+3TdGfzoOqKuP7PeM/5mpfy4T1u97n2 pu7R3u5FhntpPuO+tuv1unOftvvaR1D3aG/3IsO9NJ9xf43+Gu/c5+2+x1HUPdrbvchwL81n3MeQ +hzOey/u1Ycyur17CO7lywW+fWk/NYNU++s4ZlAP8AWPvnAPv+DwFzYAH0EbW3wHtC1l1VxzmamV tJ3Vkmm1mtsqpbCatFNpW/IKbCk469aKs6bkrHGF1hC2dZ6Pz1aHtLelyFwdq+msHtaUs7pbU9uq hElGtEJ7W4kMK2k6qxqufX8qOUtcoS0EZz1ad9aUnDWu0BrCtq4lDVu80t6WInPxrqazeljyzupu yW+rlmtUK7S3lciwkqaz6jHu+1PJWeIKbSFs6ytdaVurtK15BdYUnPWVDe1oO8sl0241t9UjI22r e0ZyVq3aPNF2Vkum1WpuqzG3t31/Km1LXoEtBWddr+asKTlrXKE1BMcFIeRrc58Ijg3kgnLfanvL R9b0lves6S1HHdtSBG+5LqjlajvLWOLYN2uis+Yl2lPyI/Qe/AgU/Qi4pCNAciOkXOImXRE8664L Srur7S0fWf7g6zuW35b/zE7T8muZ/Hkr6bX0KyLMeb3fa9RPK3jW9ztGydWuP+N93vprCBe91zvv qVyXuRfB/KtqTcA0z93/XCcJM4gP91/yWDShc6Bos9jqNQ+nfWYmbQaea7fATO6fRcuyGehMKNpM tnrNxGmfmUkPc+domEl6eCaGzDUPER5ijDUHr3FIuoqumnss5ZJPzbQapTqrJZkVVbBymmkVc+rO TESzUyUMvW5ZXjF6yyVuSypp6XQLq2FS/LYUcaOVSlh6nUc5Ld/EOS2dbln2FrzlErcllbR0ummZ k1zdtrxg1rsD7E/9GqGldo6AC3sE68ARDv1aWyGMYwRe2KvLOmCEU3+uTo7w7vrkCId+jXBd9RwB F/YI1oEjHPqF61j7MQIvbHRbBwL80K8RaonnCLiwR7AOHOHQrxFGKucIuLBHsA4c4dCf7IIR3ucX jHDq1whthHMEXNgjWAeOcOhXthKuE5O8YCPsDhjh1K8RSj0xyQt7BOvAEQ79GqEXv6JE3NZU0tbp PCvC8m1ehKXpfgk3zz2Z/02C/t1k6i3nOfc41trMrymCrj/9z49zsD/95u+3b3/+/uOf//RbI+/Z aY74D5L9KpvUuBZD7KNpGjejGGAzjmtKo+AdM02bMt7Y5IIypbzugkmYSn2sdsIakUQrhQi05lzr 7inp1O45I5xXZylp05xEW+10rXbHu2ZSxFHKlbu1kfDEIbw8n8zY40lSY3dXWx3oSXklL3EEeGgx CfePKtJKUeKQXanVHnc/phBbriGrd0kTtma0rE/6E5IB0/UiS3HObdlJyG/znH+bFQP6mDPsRi7N SQjYY6lY14ilnXhJZSOWohdWxMyh44xep5CHaBgLm9jqfDYlQECUq2KallOIEFYUaxrZTCGgDhc9 5HIpBrnEAgYgl3o4IJdDdJDT21XQ5Qg6B+yyLCsBndkp6NQSsDMtYZdGNNDhvgG6GtIBOvUH2Olc FXiJywXQUx+Anloq9NYTUODpE1Dg4T4APPQD8BI3Dma+MRHcVyjzYadYHChNJ6CkTkDJNiGp/QBK vQMFpd4hYGmSwBLPzGAp8yQoWzlAaSDt4dK7Q5poqhhSu/a9E74qELy7s4A317LBi7siWlUgWvUe iVd9MUSsvpn5eK/oIaexZNDNVgp0HsZlWG1MQl4FcSFXKYgVRgBxuYqDcGnFANxy37xJjQJ4ujT4 Lr8bvOoN8K0zKDf49nrAlzoBL8ZQ6GL2AC78ArZKIApbkwW4sFLYYkTAVucE4IIHFLbgC4AWGgXt zMUMstQIZLEkFbK5b8Cil8JVZwe4YnYAK9oKVn0fgCvmbQzaPVihI1hVIFTxRghVHdLAui8IWPFQ lWn7wbSEDpk2n0ybHHJr7Y5pBUaGWvNH6h0IMb69Yd+fFDOn++dJxH8RHPsI4MohKI5zjIbjPO/b 4ziHcSmKZ/ug4hx6UxybneCYGuJ4SmMT8ZaAZfYVJM+ZYOEJlnOqYyN59nNErFbA8gxMh2LZxhc0 s59gl21i12Ys6LV7E/TaGMRvDsxjBMFpMKMghqfcFMN5BWnE8Gw72s0aEwmKaUMU00owrO2FYfYi hjmeINhGI4ZpJQhmPyDYBCCY/YBg0wG/Ji705sAwGfhNQ6NEItgeAjDsRKB4GwiO59jXQbnugqSW mNcfJVQ944bIdA6Ui/WIuOEuVNW9EpSrO6ySrmmFdlUC7WpfJV7TCvWqH4scqGXswB0K9IvtS8nX eiJ24DgWO1ALCrZIgrRrMuIFjQNIvLq/gXq1L2OG3Dx4TQb9YobvxAzaE1GDRgZvxwmcEYgXozJK oEapV32AetUHCXaLpFj1Q5LdepKs7ewgWRMZE9QTpns4hgk6nAUKegF0u+MGIdytJeXa+KDcZCdx 397mAoiNpPtIwPcp2HTncJ1idklYCoJy4jpN2tq4TpPqPK6170K19WQS5kMGSoJZ8yaYTXNGry5k sFEEtVsCas2/oHZLi2x5T4ZhsxUMp1CPmHfKaWN4SwvDZkkMmy0IOFwOwyk0o19tL/zaGCRgk0HA YQyPZ1oKmrUtFAxfxLI9K8Eynykp2J6FkLB5U+C6Cwu4KR7ATaE0z64qCrfqzRu3Yk6ArD10QNZ0 ACxvReuVFyKEh+RsZdNKscisWQ1gtmzVAMl0WQtQnVYDoEM1gBEV6BUaqwVET67UaS0guLiWD4G1 ACVPrQWIHWsBaqXVAK0AaM6vMnJ+uT/L+A8iNQlZP3qSRseRejHn13oA74n1ALFjPUCTLdKmyUKb 6AnaVI3SplZeQJvwpxGr+jPW3BeENYeRJFlz64U1MRo503SgyC2SItkZBKnPmgS5RRKkdhZ6HCyX P2RRbVj0GS+EyYK3+awPvDWmlUBcS+eWvr4NsjyqNsMbrive1Ap4Uw+KuHYZLZoOiLvSWX3ScYA5 zdsUc00yA2zlqkM+1VJ0WDSdYLHd1Z8wCrCoPYHFOoonROZ1wCJAsdGofYFGzQiBR9oxr6pSz8QG X9u5wev9AqmYGZCq/iy3oj8gFT0VqdZXcqvG/AmFgKaFKS0EtKOSxWTfSgF1R6e7qPVmdnXUsVR3 V8c6qlW6e+P/b6dMuWUeTSzYlpAYWEjaVJPbsefKGAraqQketLmGnTQVCbiRMmW+Jk2acA9ImTIf oqZME6kK2pyGA23RdFiTpqa0KUlTDUf1avrcaVOKTSE7Q9rmCXS+1qCQnffm6DOXZICdmqZwzVPh 4Zpbc+Q532TfcM2l7/Sp7P07lyttqNKvQHU+or6BqhpJnjAHgWlO10GonCvSp8L5afq0XpWmT3h7 TJ/KpeEoE6h6eaadt3IWAXJtvtyacthFgDRzTgfTNJ+5I14n6jZ/FLF2jIiNfovc6i2i4GYf7czp brtPreeN5HlfZSN5IjhtJKdYXOSZx83FneiHuFPajDqlF1Gs0kIxe3FrRxvxZzmiz2wFKw0AGHlm fhCByNMkiTxhZXGnjM6oU9oWc8qIjDizFfh39Ml4E3NitCk2Fm2KJ0abtGK8iX4abUo/xprsZ7Gm yog2NS5FvFlcys85MtqUES3ahAaxJqJFQSy7KV5Nt9BqOg1JNbAEVjHkOwEpLAktERZtro9UUj94 8+Gw6Qpto42Sbvgob1mmg1wGm/04w0u1wxZvkm7ywweVl/CLbvFXtQKTtrHBX2elX8fABq8ebIOX QAEbPD6yxPau26tt8MNv8Lqp6gavvbHF26aOLV77couHR27xuon/ki1edbrJXy7v2ZJu83oohW1e N33Z6LmV6zbffR6vdr9go1/Rni+idg1OZZvXgXYZ1fSy1Wscxq1e7842e7PHdr/13PBNz4Jq07qT BLA67V1UtQsSIWj3x6Qp5cuSppS7O9FKKR1FVGNm8KgmbMqkcUcE1hNcGqPnUrXTjF5l5vTBkE07 ZVQdE5xqVmTVxNImWDWBR5HNn2nUloVXdftRZjVZuDU6bgWXWho18ECB656CQ3VnTAVUd8EQMN1L 9tx6nGD1Gh2e+87jbTweqhaXyVtC+WYmH4rDcs+KR0Ez3BmaVRQsw4UhWXU4eIUOKO6XgpYoNidA sXYmhikCwdYVCO71jAPcBUEwbB+TsOnY8XKJuuoWgkuKRyI2QydL/KfdwczTknXYheASJaYU/Gob +DU74eiSBK/E8pSKInm2uyJZrwPJcwT7AMC8EsczCAyK4tnG0xUcbwk4LjHZkcDUHTUp6yvI3RKj 2HHtGJZtxrDjChu58zbs7DWP0TdyZ9YwNnKpI3JLkFUryJ0j2CFAwVoibucdGWrnaAcH59H2IcDo HrVzgj4qmNqN2qzVGothtfNDVGCFHcMtvtfQ0kFLHrWjjY1a6lg2gJUVDdoOLthxBRdv52Ez9YsO t6Ulh1tmXkQtIn6gtpx5mPYUzKIfMFuiKx6wFxDLNhGr4wGzRYsFyMJUR9zCUpEKCUjVnkBqAYaJ U8wKONV+itPCz8GEbTkiMMs2EFtSL4rYomf5xOycqkWyU2d512wzqiVmoRPE2hiK2cRVCNSmYeUs elbUljI22/K4zo6uLuPaHNp5WkURp1WS9e7TqmsHCVMXXMxgH5S++cm3Wgpm3TkXUKtaHlvVbKjV 2dgJFubD86vr7VyrpXAZVpO8XyBV+VaxaiwKjlWtcWw+ODbuKMH6Gs+qVlAbz4LXhPdwuNVxgVvT Kd9eruQ1pe751p8DkOnJt+NgW7Ei23Iuxra8Z6DYtODe6ziCLcyuBMlHRlZCDffMq2wrEpAbysG1 xaIEbQtq1/COa6P78Gqm52fNYIYXHqsqClajVQmIVtUKWmPfAa0JBCc7ApwqEJrsCmjG7ssB06MP AvYEAE8M9Pa3AfcJW5sPZBNs6tmXB3pwBJt6tIRN2wxq2U+CWrQR0qKXhrTUSHmAGga3kBDa9nht kKqGwW1nqIbgFuNpaKs6hLZdTpgR2pqGoe01LCRgW8NaWCGQZZslgs5yDYoE+pS0TNCzCwi2JIWC rvVahrPdlQrYZqFA54lCQXdfBqhGCwVdvtPgWQFDbDstUFnC2TGUKiWc3aIeJQx/AGtlpH2UYHqc Y6FaxqJB1BKWlg2OswWn1xLXcCdZ1pknWbzFfZYVPaZtsMfUrE7/O7AdOzyI/S6s9V9pGeUY6apM 0nVnWqZ7h3QvV4C4p1yVECoILRnh6gkEQ9zjXMsCaKXcI8jtx+cDW0virV8l3iPo7UfQO+l1hxCh HScNJ/VW+/KlxOhKYSUQYiTimjwRQwIRiycScXVlMKYfoGGQoJJwCnVjWwkSIW8cJz+P4fiZOrJz Sm2zc0p63Ap+Rlfj58sj2rRk6FTbVxj6TNScHhx97YLt+vPlxzluwGxR5Jp3VfpcMV++f/nhX9hr of7dXjuK3iQfgizga1T+PmC+nqSirqb/5yf0qb3O7Lyfv6FfS6rfrgm1PiMj+XlXuH3+8NP+MdFe pzqJ1pufROv6bf5zvif2Q+Xv9+PyHT845/HOeYndOy9WMH/OeU6vk0fgPInz78z517/J0wnFGvyE ID4/obk0dD55zSftX3Xdv4jWezLHIjzttk0YN/40Yd7xD3/99Ol3H//rw5foHsYv/VZGZ9fA55hd i+VXzK7G175+ZrFmV28ff5K5pQ/vY7Qt1jHnWYLFZ51PzhuDP7xs5vy79zHawtW383D1553X9Q8t zMRLnF+3D3/9/PhG3j7/5Fxq199myWwoPj+fSYK5Ncynr/l8BZ/791DwvX8O9ZzvdXxcuTbG7X9/ /v7v/mF89VRDZ5T0l0mYUbIfJj03oxnjXoX/9sftwzGhB0jWEKr3HUL9Nb7L3J+H/h42ivP0PiRn bHKZaxGedzzjrPXxsThOcz3kD+4tvAPDUtrY/ksbv8K//MsrCf7z8v/d+xCcwWLcfpfwvN/179l0 LMVJZh9/Sh++ulM8gC/zgy2ZS8ZXRE/OJc9wr11YhnER4kFHd88gIzI2z/p9zrO+69Q1LMO4+DB9 fB93OQ8fK1B83vdMyUIFF8/davr+y373POG5qq+QJ349gUS4Xv4bzSqVXCTCtflEuPZuiXAd9mul ed0nwhUJgyTC9fhuEyMjEWZ7JQ4zxXGJA30iCcbImgTDP1JgzFmTXkqS9MK/Jr1VKls4ycFskADz OhPgus8nqdHkFzNA6tuSneJwLE174RNpL8bStJeSJL2NKRrSXvjRpJeSnOAU/BqJJziFp1E4Hy/2 6aWckGsx0s7IYcuEFx+72Bm5jsR0l12R7hb5IbkluzNN8ckuKsmW6tISqS51SHTNThNd1eKn4hEM +/uX/wNS7q9kZW5kc3RyZWFtCmVuZG9iago2IDAgb2JqCjQ5NTcKZW5kb2JqCjQgMCBvYmoKPDwv VHlwZS9QYWdlL01lZGlhQm94IFswIDAgMTMwMiA4NDVdCi9Sb3RhdGUgMC9QYXJlbnQgMyAwIFIK L1Jlc291cmNlczw8L1Byb2NTZXRbL1BERiAvVGV4dF0KL0V4dEdTdGF0ZSA5IDAgUgovRm9udCAx MCAwIFIKPj4KL0NvbnRlbnRzIDUgMCBSCj4+CmVuZG9iagozIDAgb2JqCjw8IC9UeXBlIC9QYWdl cyAvS2lkcyBbCjQgMCBSCl0gL0NvdW50IDEKPj4KZW5kb2JqCjEgMCBvYmoKPDwvVHlwZSAvQ2F0 YWxvZyAvUGFnZXMgMyAwIFIKL01ldGFkYXRhIDEyIDAgUgo+PgplbmRvYmoKNyAwIG9iago8PC9U eXBlL0V4dEdTdGF0ZQovT1BNIDE+PmVuZG9iago5IDAgb2JqCjw8L1I3CjcgMCBSPj4KZW5kb2Jq CjEwIDAgb2JqCjw8L1I4CjggMCBSPj4KZW5kb2JqCjggMCBvYmoKPDwvQmFzZUZvbnQvSGVsdmV0 aWNhL1R5cGUvRm9udAovRW5jb2RpbmcgMTEgMCBSL1N1YnR5cGUvVHlwZTE+PgplbmRvYmoKMTEg MCBvYmoKPDwvVHlwZS9FbmNvZGluZy9EaWZmZXJlbmNlc1sKNDUvbWludXNdPj4KZW5kb2JqCjEy IDAgb2JqCjw8L1R5cGUvTWV0YWRhdGEKL1N1YnR5cGUvWE1ML0xlbmd0aCAxMzY0Pj5zdHJlYW0K PD94cGFja2V0IGJlZ2luPSfvu78nIGlkPSdXNU0wTXBDZWhpSHpyZVN6TlRjemtjOWQnPz4KPD9h ZG9iZS14YXAtZmlsdGVycyBlc2M9IkNSTEYiPz4KPHg6eG1wbWV0YSB4bWxuczp4PSdhZG9iZTpu czptZXRhLycgeDp4bXB0az0nWE1QIHRvb2xraXQgMi45LjEtMTMsIGZyYW1ld29yayAxLjYnPgo8 cmRmOlJERiB4bWxuczpyZGY9J2h0dHA6Ly93d3cudzMub3JnLzE5OTkvMDIvMjItcmRmLXN5bnRh eC1ucyMnIHhtbG5zOmlYPSdodHRwOi8vbnMuYWRvYmUuY29tL2lYLzEuMC8nPgo8cmRmOkRlc2Ny aXB0aW9uIHJkZjphYm91dD0ndXVpZDplMWQ2N2JjNC1lOWYxLTExZWQtMDAwMC1iNGJmNWVhM2M5 NWQnIHhtbG5zOnBkZj0naHR0cDovL25zLmFkb2JlLmNvbS9wZGYvMS4zLycgcGRmOlByb2R1Y2Vy PSdHUEwgR2hvc3RzY3JpcHQgOS4wNicvPgo8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0ndXVp ZDplMWQ2N2JjNC1lOWYxLTExZWQtMDAwMC1iNGJmNWVhM2M5NWQnIHhtbG5zOnhtcD0naHR0cDov L25zLmFkb2JlLmNvbS94YXAvMS4wLyc+PHhtcDpNb2RpZnlEYXRlPjIwMTMtMDQtMzBUMjI6MzQ6 MTkrMDI6MDA8L3htcDpNb2RpZnlEYXRlPgo8eG1wOkNyZWF0ZURhdGU+MjAxMy0wNC0zMFQyMjoz NDoxOSswMjowMDwveG1wOkNyZWF0ZURhdGU+Cjx4bXA6Q3JlYXRvclRvb2w+R05VIGxpYnBsb3Qg ZHJhd2luZyBsaWJyYXJ5IDQuNDwveG1wOkNyZWF0b3JUb29sPjwvcmRmOkRlc2NyaXB0aW9uPgo8 cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0ndXVpZDplMWQ2N2JjNC1lOWYxLTExZWQtMDAwMC1i NGJmNWVhM2M5NWQnIHhtbG5zOnhhcE1NPSdodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAvbW0v JyB4YXBNTTpEb2N1bWVudElEPSd1dWlkOmUxZDY3YmM0LWU5ZjEtMTFlZC0wMDAwLWI0YmY1ZWEz Yzk1ZCcvPgo8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0ndXVpZDplMWQ2N2JjNC1lOWYxLTEx ZWQtMDAwMC1iNGJmNWVhM2M5NWQnIHhtbG5zOmRjPSdodHRwOi8vcHVybC5vcmcvZGMvZWxlbWVu dHMvMS4xLycgZGM6Zm9ybWF0PSdhcHBsaWNhdGlvbi9wZGYnPjxkYzp0aXRsZT48cmRmOkFsdD48 cmRmOmxpIHhtbDpsYW5nPSd4LWRlZmF1bHQnPlBvc3RTY3JpcHQgcGxvdDwvcmRmOmxpPjwvcmRm OkFsdD48L2RjOnRpdGxlPjwvcmRmOkRlc2NyaXB0aW9uPgo8L3JkZjpSREY+CjwveDp4bXBtZXRh PgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCjw/eHBhY2tldCBlbmQ9J3cnPz4KZW5k c3RyZWFtCmVuZG9iagoyIDAgb2JqCjw8L1Byb2R1Y2VyKEdQTCBHaG9zdHNjcmlwdCA5LjA2KQov Q3JlYXRpb25EYXRlKEQ6MjAxMzA0MzAyMjM0MTkrMDInMDAnKQovTW9kRGF0ZShEOjIwMTMwNDMw MjIzNDE5KzAyJzAwJykKL0NyZWF0b3IoR05VIGxpYnBsb3QgZHJhd2luZyBsaWJyYXJ5IDQuNCkK L1RpdGxlKFBvc3RTY3JpcHQgcGxvdCk+PmVuZG9iagp4cmVmCjAgMTMKMDAwMDAwMDAwMCA2NTUz NSBmIAowMDAwMDA1MjgxIDAwMDAwIG4gCjAwMDAwMDcwMjUgMDAwMDAgbiAKMDAwMDAwNTIyMiAw MDAwMCBuIAowMDAwMDA1MDYyIDAwMDAwIG4gCjAwMDAwMDAwMTUgMDAwMDAgbiAKMDAwMDAwNTA0 MiAwMDAwMCBuIAowMDAwMDA1MzQ2IDAwMDAwIG4gCjAwMDAwMDU0NDYgMDAwMDAgbiAKMDAwMDAw NTM4NyAwMDAwMCBuIAowMDAwMDA1NDE2IDAwMDAwIG4gCjAwMDAwMDU1MjYgMDAwMDAgbiAKMDAw MDAwNTU4NCAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDEzIC9Sb290IDEgMCBSIC9JbmZvIDIg MCBSCi9JRCBbPDgzRjUxQ0JCRTk2RERFNTQyRUQyMUYxOTA4M0JCQTI0Pjw4M0Y1MUNCQkU5NkRE RTU0MkVEMjFGMTkwODNCQkEyND5dCj4+CnN0YXJ0eHJlZgo3MjE0CiUlRU9GCg== --Apple-Mail-87-361677366--