From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Andy Wingo Newsgroups: gmane.lisp.guile.devel Subject: review/merge request: wip-array-refactor Date: Sun, 19 Jul 2009 15:59:10 +0200 Message-ID: NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1248011960 9110 80.91.229.12 (19 Jul 2009 13:59:20 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 19 Jul 2009 13:59:20 +0000 (UTC) To: guile-devel Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Sun Jul 19 15:59:12 2009 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1MSWv2-0003YL-1i for guile-devel@m.gmane.org; Sun, 19 Jul 2009 15:59:12 +0200 Original-Received: from localhost ([127.0.0.1]:57949 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1MSWv1-0000Tj-Is for guile-devel@m.gmane.org; Sun, 19 Jul 2009 09:59:11 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1MSWus-0000SX-PP for guile-devel@gnu.org; Sun, 19 Jul 2009 09:59:02 -0400 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1MSWun-0000Hv-44 for guile-devel@gnu.org; Sun, 19 Jul 2009 09:59:01 -0400 Original-Received: from [199.232.76.173] (port=34572 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1MSWum-0000HY-PU for guile-devel@gnu.org; Sun, 19 Jul 2009 09:58:56 -0400 Original-Received: from a-sasl-quonix.sasl.smtp.pobox.com ([208.72.237.25]:35635 helo=sasl.smtp.pobox.com) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1MSWum-0008Sj-5e for guile-devel@gnu.org; Sun, 19 Jul 2009 09:58:56 -0400 Original-Received: from localhost.localdomain (unknown [127.0.0.1]) by a-sasl-quonix.sasl.smtp.pobox.com (Postfix) with ESMTP id 56C5AA000 for ; Sun, 19 Jul 2009 09:58:55 -0400 (EDT) Original-Received: from unquote (unknown [83.32.66.205]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by a-sasl-quonix.sasl.smtp.pobox.com (Postfix) with ESMTPSA id 0D0919FFF for ; Sun, 19 Jul 2009 09:58:53 -0400 (EDT) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.92 (gnu/linux) X-Pobox-Relay-ID: 4C524C2C-746C-11DE-9EDB-F699A5B33865-02397024!a-sasl-quonix.pobox.com X-detected-operating-system: by monty-python.gnu.org: Solaris 10 (beta) X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:8886 Archived-At: Hi all, I've finished up my refactor of Guile's arrays. To my eye it's much nicer now. The only bits I could anticipate being controversial would be the last two or three patches, in which bytevectors are given an "element type" field. This is so that I can make the srfi-4 uniform vector code use bytevectors as their representation -- it's one less orthogonal type that the VM would have to deal with. Those patches also allow uniform vectors of one type to be accessed using accessors for other types: (u8vector-ref #u32(#xffffffff) 0) => 255 Although: (u8vector? #u32(#xffffffff)) => #f But: (bytevector? #u32(#xffffffff)) => #t Anyway, logs are here, in chronological order. Let me know what you think. Make check should pass at each of these patches. This is on the "wip-array-refactor" branch, which I had been rebasing, but I don't plan to rebase it again. Andy commit 86d88a223c64276e7cd9b4503e7e2ecca5aae320 Author: Andy Wingo Date: Thu Jul 16 21:51:47 2009 +0200 remove deprecated functions from unif.c * libguile/unif.h: * libguile/unif.c: Remove deprecated functions. * module/ice-9/deprecated.scm: Remove array-related deprecated functions. * NEWS: Update. commit 4b126598445c4f12c0aebca2adfaa45f3edd86ab Author: Andy Wingo Date: Thu Jul 16 22:02:25 2009 +0200 remove convert.{c,i.c,h} * libguile/convert.c: * libguile/convert.h: * libguile/convert.i.c: Remove these functions, which were undocumented, not in the libguile/ header, and thus unlikely to have been used. commit a4a0d399c877cb802cdaf2c48713d3377a412c4f Author: Andy Wingo Date: Thu Jul 16 22:15:04 2009 +0200 clean up libguile/Makefile.am * libguile/Makefile.am: Clean up some of the file lists, should make future diffs easier to parse. commit b6149d8d9f35c8091a31b12fb3aeecee0e3a470c Author: Andy Wingo Date: Fri Jul 17 00:16:43 2009 +0200 rename scm_i_make_ra to scm_i_make_array * libguile/unif.c (scm_i_make_array): Rename from scm_i_make_ra. All callers changed. commit 5d1b3b2db9349b615baac313ae5a111fa68573ac Author: Andy Wingo Date: Fri Jul 17 00:25:49 2009 +0200 rename ramap.[ch] to array-map.[ch] * libguile/array-map.c: * libguile/array-map.h: Rename from ramap.c and ramap.h. * libguile.h: * libguile/Makefile.am: * libguile/eq.c: * libguile/init.c: * libguile/sort.c: * libguile/unif.c: * libguile/vectors.c: All referrers changed. commit c53c0893a3bad3312230003707f71c2f441460d4 Author: Andy Wingo Date: Fri Jul 17 00:47:31 2009 +0200 parts of unif.[ch] to array-handle.[ch] * libguile/array-handle.c: * libguile/array-handle.h: Move some parts of unif.c and unif.h to these new files. * libguile/unif.c: * libguile/unif.h: Update includers. Since unif.h depends on the array handle type, we include array-handle.h, which also means that there will be no difference for our callers. * libguile/init.c: Call scm_init_array_handle, though it does nothing as of yet. * libguile/Makefile.am: Adapt for new files. commit cf396142405d9076cc20eca9bf53376e80359a56 Author: Andy Wingo Date: Fri Jul 17 00:58:32 2009 +0200 bitvector exodus from unif.[ch] * libguile/Makefile.am: * libguile/unif.c: * libguile/unif.h: * libguile/bitvectors.c: * libguile/bitvectors.h: Move bitvector functionality out of unif.[ch]. * libguile/array-handle.c: * libguile/array-map.c: * libguile/init.c: * libguile/read.c: * libguile/srfi-4.c: * libguile/vectors.c: Oh, what a tangled web we weave... commit 2fa901a51f62da8a01112aefbf687530f4bff160 Author: Andy Wingo Date: Fri Jul 17 01:08:35 2009 +0200 rename unif.[ch] to arrays.[ch] * libguile/Makefile.am: * libguile/unif.c: * libguile/unif.h: * libguile/arrays.c: * libguile/arrays.h: Rename unif.[ch] to arrays.[ch]. * libguile.h: * libguile/array-handle.c: * libguile/array-map.c: * libguile/bitvectors.c: * libguile/bytevectors.c: * libguile/eq.c: * libguile/gc-card.c: * libguile/gc-malloc.c: * libguile/gc-mark.c: * libguile/gc.c: * libguile/init.c: * libguile/inline.h: * libguile/print.c: * libguile/random.c: * libguile/read.c: * libguile/socket.c: * libguile/sort.c: * libguile/srfi-4.c: * libguile/srfi-4.h: * libguile/strports.c: * libguile/vectors.c: * libguile/vectors.h: Update includers. commit 2a610be59412a9d633a373c6f6ec4d4794c40fd8 Author: Andy Wingo Date: Sun Jul 19 15:04:40 2009 +0200 add generic array implementation facility * libguile/array-handle.c (scm_i_register_array_implementation): (scm_i_array_implementation_for_obj): Add generic array facility, which will (in a few commits) detangle the array code. (scm_array_get_handle): Use the generic array facility. Note that scm_t_array_handle no longer has ref and set function pointers; instead it has a pointer to the array implementation. It is unlikely that code out there used these functions, however, as the supported way was through scm_array_handle_ref/set_x. (scm_array_handle_pos): Move this function here from arrays.c. (scm_array_handle_element_type): New function, returns a Scheme value representing the type of element stored in this array. * libguile/array-handle.h (scm_t_array_element_type): New enum, for generically determining the type of an array. (scm_array_handle_rank): (scm_array_handle_dims): These are now just #defines. * libguile/arrays.c: * libguile/bitvectors.c: * libguile/bytevectors.c: * libguile/srfi-4.c: * libguile/strings.c: * libguile/vectors.c: Register array implementations for all of these. * libguile/inline.h: Update for array_handle_ref/set change. * libguile/deprecated.h: Need to include arrays.h now. commit 66b9d7d304a349d5bb4f763a47143f09da58d97f Author: Andy Wingo Date: Fri Jul 17 12:45:24 2009 +0200 remove enclosed arrays * libguile/arrays.h: * libguile/array-map.c: * libguile/arrays.c: * libguile/deprecated.c: Remove "enclosed arrays". The only user-facing procedures that this affects are scm_enclose_array / enclose-array. If enclosed arrays are added back, it should be through the generic array interface; but really, it sounds like something that would be better implemented in Scheme. commit 1030b45049f564f4abd459abd8e59db34c7867cc Author: Andy Wingo Date: Fri Jul 17 18:54:06 2009 +0200 move generic array foo out to its own file * libguile/arrays.h: * libguile/arrays.c: * libguile/generalized-arrays.h: * libguile/generalized-arrays.c: Move some generic functionality out of arrays.c to a new file. * libguile/array-map.c: * libguile/deprecated.c: * libguile/init.c: Update includers. commit f332e9571703ddcd27c51ebe3c847459c2a649b7 Author: Andy Wingo Date: Fri Jul 17 19:05:32 2009 +0200 generic vector ops to own file * libguile/Makefile.am: * libguile/vectors.c: * libguile/vectors.h: * libguile/generalized-vectors.c: * libguile/generalized-vectors.h: Move generic vector ops off into their own file too. The implementation is now based on the generic array-handle infrastructure. * libguile.h: * libguile/array-map.c: * libguile/bitvectors.c: * libguile/random.c: * libguile/srfi-4.c: Update includers. commit 476b894c71b436f3befb8af46b899aaf244763e2 Author: Andy Wingo Date: Sat Jul 18 12:18:15 2009 +0200 uniform vector functions to their own file * libguile/uniform.c: * libguile/uniform.h: * libguile/srfi-4.c: * libguile/srfi-4.h: * libguile/Makefile.am: Move uniform vector funcs out of srfi-4 to their own file. * libguile.h: * libguile/arrays.c: * libguile/bytevectors.c: Update includers. commit f45eccffa73c043466a4cc0f5037132ee5795eee Author: Andy Wingo Date: Sat Jul 18 12:43:54 2009 +0200 add registry of vector constructors, make-generalized-vector * libguile/generalized-vectors.h: * libguile/generalized-vectors.c: Add a registry of vector constructors. (scm_make_generalized_vector): New public function, constructs a vector of a given type. * libguile/bitvectors.c: * libguile/bytevectors.c: * libguile/srfi-4.c: * libguile/strings.c: * libguile/vectors.c: Register vector constructors. * libguile/extensions.c (scm_init_extensions): No need to NULL the list of registered extensions here, the static init does it for us. Allows scm_c_register_extension to be called before scm_init_extensions. * libguile/init.c (scm_i_init_guile): Move array initialization earlier, so e.g. scm_init_strings has access to a valid list of array element types when registering its vector constructor. commit 943a0a8759504c4a367c1904bef4a8afbc6208dd Author: Andy Wingo Date: Sat Jul 18 12:58:37 2009 +0200 make-typed-array builds backing vector via make-generalized-vector * libguile/arrays.c: Rework to use scm_make_generalized_vector instead of our own type table. * libguile/bitvectors.c: Fix some includes. commit ac8ed3db31769d7ede5e75fba1697e8dde55fae4 Author: Andy Wingo Date: Sat Jul 18 13:26:18 2009 +0200 any->u8vector and family now implemented in Scheme * module/Makefile.am: * module/srfi/srfi-4/gnu.scm: New module, for extensions to srfi-4. Currently defines the any->FOOvector family. * libguile/srfi-4.c: * libguile/srfi-4.i.c: Dispatch scm_any_to_FOOvector calls to the scheme-implemented functions in (srfi srfi-4 gnu). commit f332089ed43761440a2a8c272ee61a709b38cc24 Author: Andy Wingo Date: Sat Jul 18 13:46:29 2009 +0200 bytevector inlinedness indicated by flag, not length * libguile/bytevectors.h (SCM_BYTEVECTOR_INLINE_P): Change to check a flag instead of checking the length of the bytevector. * libguile/bytevectors.c (make_bytevector_from_buffer): Handle the len <= inline threshold case as well. Set the inline flag as appropriate. (make_bytevector): Updat the inline flag as appropriate. (scm_c_take_bytevector): Just dispatch to make_bytevector_from_buffer. (scm_i_shrink_bytevector): Update the inline flag as appropriate. Update the length when shrinking an already-inlined vector. (STRING_TO_UTF): Fix some indentation. commit e286c973fcd63c0930d9302cc5f1a280b9b22615 Author: Andy Wingo Date: Sun Jul 19 15:11:53 2009 +0200 bytevectors have "element type" field, e.g. for generalized-vector-ref Bytevectors have a very close relationship to other forms of uniform vectors. Often you want to view a u64vector as a series of bytes, for writing over a socket; or to process an incoming stream using the convenient and less error-prone s16vector-ref API rather than bytevector-s16-native-ref. The essential needs of the representation of a bytevector and an s64vector are the same, so we take advantage of that and extend the bytevector implementation to have a "native type" field, which defaults to VU8. This commit doesn't actually expose any user-noticeable changes, however. * libguile/bytevectors.h (SCM_BYTEVECTOR_ELEMENT_TYPE): New internal defines. (scm_i_make_typed_bytevector, scm_c_take_typed_bytevector): New internal functions. * libguile/bytevectors.c (SCM_BYTEVECTOR_SET_ELEMENT_TYPE): (SCM_BYTEVECTOR_TYPE_SIZE): (SCM_BYTEVECTOR_TYPED_LENGTH): New internal macros. (make_bytevector, make_bytevector_from_buffer): Take an extra argument, the element type. The length argument is interpreted as being the number of elements, which corresponds to the number of bytes in the default VU8 case. Doing it this way eliminates a class of bugs -- e.g. a u32vector of length 3 bytes doesn't make sense. We do have to check for another class of bugs: overflow. The length stored on the bytevector itself is still the byte length, though. (scm_i_make_typed_bytevector): (scm_c_take_typed_bytevector): New internal functions. (scm_i_shrink_bytevector): Make sure the new size is valid for the bytevector's type. (scm_i_bytevector_generalized_set_x): Remove this function, the array-handle infrastructure takes care of this for us. (print_bytevector): Print the bytevector according to its type. (scm_make_bytevector, scm_bytevector_copy) (scm_uniform_array_to_bytevector) (scm_u8_list_to_bytevector, scm_bytevector_to_uint_list): Adapt to make_bytevector extra arg. (bv_handle_ref, bv_handle_set_x): Adapt to ref and set based on the type of the bytevector, e.g. f64 or u8. (bytevector_get_handle): Set the typed length of the vector, not the byte length. Conflicts: libguile/bytevectors.c commit cd43fdc5b7a7c851ee0f2b4e96a1f394fb50d869 Author: Andy Wingo Date: Sat Jul 18 19:03:28 2009 +0200 fix (bytevector-ieee-single-native-set! x 0 0) * libguile/bytevectors.c (VALIDATE_REAL): SCM_VALIDATE_REAL is not what we need for checking values for bytevector-ieee-single-native-set! et al, so define our own validator. (IEEE754_SET, IEEE754_NATIVE_SET): Use it. commit 6f4895577734bb107d488f31dca82d5ad4c25a65 Author: Andy Wingo Date: Sun Jul 19 15:35:33 2009 +0200 reimplement srfi-4 vectors on top of bytevectors * libguile/srfi-4.h: * libguile/srfi-4.c (scm_make_srfi_4_vector): New function, exported by (srfi srfi-4 gnu). * libguile/srfi-4.i.c: Removed. * module/srfi/srfi-4.scm: * module/srfi/srfi-4/gnu.scm: Reimplement srfi-4 vectors on top of bytevectors. The implementation is mostly in Scheme now. * module/rnrs/bytevector.scm: * libguile/bytevectors.c (scm_uniform_array_to_bytevector): No more need for this, as uniform vectors are bytevectors, and we can get the backing vector of an array via shared-array-root. * libguile/bytevectors.c (bytevector_ref_c32, bytevector_ref_c64) (bytevector_set_c32, bytevector_set_c64): Fix some embarrassing bugs. Still need to do an upper bounds check. * libguile/deprecated.h: Remove deprecated array functions: scm_i_arrayp, scm_i_array_ndim, scm_i_array_mem, scm_i_array_v, scm_i_array_base, scm_i_array_dims, and the deprecated macros: SCM_ARRAYP, SCM_ARRAY_NDIM, SCM_ARRAY_CONTP, SCM_ARRAY_MEM, SCM_ARRAY_V, SCM_ARRAY_BASE, SCM_ARRAY_DIMS. * libguile/deprecated.c (scm_uniform_vector_read_x) (scm_uniform_vector_write, scm_uniform_array_read_x) (scm_uniform_array_write): Newly deprecated functions. * libguile/generalized-arrays.c (scm_array_type): Remove the bytevector hack. This does introduce the bug that #vu8(1 2 3) will compile to #u8(1 2 3). I'm working on that. * libguile/objcodes.c (scm_bytecode_to_objcode, scm_objcode_to_bytecode): Rework to operate on bytevectors, as scm_make_u8vector now causes a module lookup, which can't be done e.g. when loading the VM boot program for psyntax-pp.go on a fresh bootstrap. * libguile/objcodes.h (SCM_F_OBJCODE_IS_BYTEVECTOR): (SCM_OBJCODE_IS_BYTEVECTOR): s/U8VECTOR/BYTEVECTOR/. * module/ice-9/boot-9.scm (the-scm-module): A terrible hack to pull in (srfi srfi-4), as the bindings are primarily there now. We'll worry about this later. * module/language/glil/compile-assembly.scm (dump-object): Update to work with array-contents and shared-array-root. * test-suite/tests/bytevectors.test: Remove uniform-array->bytevector test. commit ae3543b1f27e973b25060f71f095bf23ef149aee Author: Andy Wingo Date: Sat Jul 18 19:08:43 2009 +0200 clean up includes in vectors.[ch] * libguile/vectors.h: * libguile/vectors.c: Clean up the includes... mostly. -- http://wingolog.org/