0.9.16.17:

	Support for external formats in SB-ALIEN. The C-STRING alien-type
        specifier now accepts :EXTERNAL-FORMAT and :ELEMENT-TYPE parameters.
        This is a slightly incompatible change: to get the behaviour of
        the old C-STRING alien-type, use (C-STRING :EXTERNAL-FORMAT :ASCII
        :ELEMENT-TYPE BASE-CHAR).

        Thanks to Yaroslav Kavenchuk for doing most of the work on this.

        * Also add support for non-ascii pathnames
        * Add some recent CONTRIBUTORS
        * Update INSTALL
        * Add argument quote/space escaping to RUN-PROGRAM on win32

CREDITS

@@ -606,6 +606,11 @@ Teemu Kalvas:
  character database, restoring the FAST-READ-CHAR optimization and
  developing external format support.
 
+Yaroslav Kavenchuk:
+ He implemented several missing features and fixed many bugs in
+ the win32 port. He also worked on external-format support for
+ SB-ALIEN.
+
 Frederik Kuivinen:
  He showed how to implement the DEBUG-RETURN functionality.
 
@@ -673,6 +678,9 @@ William ("Bill") Newman:
  updating documentation, and even, for better or worse, getting
  rid of various functionality (e.g. the byte interpreter).
 
+NIIMI Satoshi:
+ He contributed a number of fixes to the FreeBSD port.
+
 Patrik Nordebo:
  He contributed to the port of SBCL to MacOS X, finding solutions for
  ABI and assembly syntax differences between Darwin and Linux.
@@ -684,6 +692,9 @@ Luís Oliveira:
 Scott Parish:
  He ported SBCL to OpenBSD-with-ELF.
 
+Timothy Ritchey:
+ He implemented SB-BSD-SOCKETS support for the win32 port.
+
 Kevin M. Rosenberg:
  He provided the ACL-style toplevel (sb-aclrepl contrib module), and
  a number of MOP-related bug reports. He also creates the official

INSTALL

@@ -154,7 +154,8 @@ INSTALLING SBCL
  (pushnew x features))
  (disable (x)
  (setf features (remove x features))))
- ;; Threading support, available on x86/x86-64 Linux only.
+ ;; Threading support, available only on x86/x86-64 Linux, x86 Solaris
+ ;; and x86 Mac OS X (experimental).
  (enable :sb-thread)))
 
  This is the preferred way because it lets local changes interact
@@ -241,11 +242,12 @@ INSTALLING SBCL
  NetBSD X
  Solaris X X
  Tru64 X
- Darwin (Mac OS X) X
+ Darwin (Mac OS X) X X
+ Windows X
 
  Some operating systems are more equal than others: most of the
- development and testing is done on x86 Linux and *BSD, PPC Linux
- and Mac OS X.
+ development and testing is done on x86/x86-64 Linux and x86/PPC
+ Mac OS X.
 
  If an underprivileged platform is important to you, you can help
  by e.g. testing during the monthly freeze periods, and most

NEWS

@@ -1,5 +1,13 @@
 ;;;; -*- coding: utf-8; -*-
 changes in sbcl-0.9.17 (0.9.99?) relative to sbcl-0.9.16:
+ * incompatible change: External-format support for FFI calls. The
+ SB-ALIEN:C-STRING no longer implies an ASCII
+ external-format. Instead the string is subject to external-format
+ conversion. Additionally return values of type SB-ALIEN:C-STRING
+ are by default of type (SIMPLE-ARRAY CHARACTER), not
+ SIMPLE-BASE-STRING. For an alien type that behaves like the old
+ SB-ALIEN:C-STRING type, use (SB-ALIEN:C-STRING :EXTERNAL-FORMAT
+ :ASCII :ELEMENT-TYPE BASE-CHAR). (thanks to Yaroslav Kavenchuk)
  * incompatible change: SB-EXT package no longer contains the
  following unused symbols: *GC-NOTIFY-AFTER*, *GC-NOTIFY-BEFORE*,
  *GC-NOTIFY-STREAM*, *ERROR-PRINT-LENGTH*, *ERROR-PRINT-LEVEL*,
@@ -16,6 +24,8 @@ changes in sbcl-0.9.17 (0.9.99?) relative to sbcl-0.9.16:
  Slobodov)
  * bug fix: better detection of circularities in the file-compiler.
  (reported by Marco Monteiro)
+ * bug fix: the CL pathname functions now work with files that have
+ non-ASCII characters in their names (thanks to Yaroslav Kavenchuk)
  * bug fix: The :PTY argument for RUN-PROGRAM will now work on 
  systems with Unix98 pty semantics.
  * bug fix: ASDF-INSTALL will now work with bsd tar.
@@ -192,6 +202,8 @@ changes in sbcl-0.9.13 relative to sbcl-0.9.12:
  faster
  * optimization: added a limited bytecode compiler for simple toplevel
  forms, speeding up compilation and FASL loading
+ * bug fix: the statistical profiler now properly distinguishes anonymous
+ functions
 
 changes in sbcl-0.9.12 relative to sbcl-0.9.11:
  * minor incompatible change: in sbcl-0.9.11 (but not earlier

contrib/sb-simple-streams/file.lisp

@@ -115,8 +115,8 @@
 ;;; TODO: use this in src/code/fd-stream.lisp:fd-stream-misc-routine
 ;;; as well, snarf error reporting from there.
 (defun revert-file (filename original)
- (declare (type simple-base-string filename)
- (type (or simple-base-string null) original))
+ (declare (type simple-string filename)
+ (type (or simple-string null) original))
  ;; We can't do anything unless we know what file were
  ;; dealing with, and we don't want to do anything
  ;; strange unless we were writing to the file.
@@ -140,8 +140,8 @@
 ;;; TODO: use this in src/code/fd-stream.lisp:fd-stream-misc-routine
 ;;; as well, snarf error reporting from there.
 (defun delete-original (filename original)
- (declare (type simple-base-string filename)
- (type (or simple-base-string null) original))
+ (declare (type simple-string filename)
+ (type (or simple-string null) original))
  (when original
  (multiple-value-bind (okay err) (sb-unix:unix-unlink original)
  (unless okay

doc/manual/ffi.texinfo

@@ -283,15 +283,37 @@ types to declare that no useful value is returned. Using
 return zero values.
 
 @item
-The foreign type specifier @code{sb-alien:c-string} is similar to
-@code{(* char)}, but is interpreted as a null-terminated string, and is
-automatically converted into a Lisp string when accessed; or if the
-pointer is C @code{NULL} or @code{0}, then accessing it gives Lisp
-@code{nil}. Lisp strings of type @code{base-string} are stored with a
-trailing NUL termination, so no copying (either by the user or the
-implementation) is necessary when passing them to foreign code; strings
-of type @code{(simple-array character (*))} are copied by the
-implementation as required.
+The foreign type specifier @code{(sb-alien:c-string &key external-format
+element-type)} is similar to @code{(* char)}, but is interpreted as a
+null-terminated string, and is automatically converted into a Lisp
+string when accessed; or if the pointer is C @code{NULL} or @code{0},
+then accessing it gives Lisp @code{nil}.
+
+External format conversion is automatically done when Lisp strings are
+passed to foreign code, or when foreign strings are passed to Lisp code.
+If the type specifier has an explicit @code{external-format}, that
+external format will be used. Otherwise a default external format that
+has been determined at SBCL startup time based on the current locale
+settings will be used. For example, when the following alien routine is
+called, the Lisp string given as argument is converted to an
+@code{ebcdic} octet representation.
+
+@lisp
+(define-alien-routine test int (str (c-string :external-format :ebcdic-us)))
+@end lisp
+
+Lisp strings of type @code{base-string} are stored with a trailing NUL
+termination, so no copying (either by the user or the implementation) is
+necessary when passing them to foreign code, assuming that the
+@code{external-format} and @code{element-type} of the @code{c-string}
+type are compatible with the internal representation of the string. For
+an SBCL built with Unicode support that means an @code{external-format}
+of @code{:ascii} and an @code{element-type} of @code{base-char}. Without
+Unicode support the @code{external-format} can also be
+@code{:iso-8859-1}, and the @code{element-type} can also be
+@code{character}. If the @code{external-format} or @code{element-type}
+is not compatible, or the string is a @code{(simple-array character
+(*))}, this data is copied by the implementation as required.
 
 Assigning a Lisp string to a @code{c-string} structure field or
 variable stores the contents of the string to the memory already

package-data-list.lisp-expr

@@ -831,6 +831,8 @@ retained, possibly temporariliy, because it might be used internally."
  "CHARACTER-DECODING-ERROR" "CHARACTER-DECODING-ERROR-OCTETS"
  "CHARACTER-ENCODING-ERROR" "CHARACTER-ENCODING-ERROR-CODE"
  "STREAM-DECODING-ERROR" "STREAM-ENCODING-ERROR"
+ "C-STRING-ENCODING-ERROR" "C-STRING-ENCODING-ERROR-EXTERNAL-FORMAT"
+ "C-STRING-DECODING-ERROR" "C-STRING-DECODING-ERROR-EXTERNAL-FORMAT"
  "ATTEMPT-RESYNC" "FORCE-END-OF-FILE"
 
  ;; bootstrapping magic, to make things happen both in
@@ -2090,6 +2092,7 @@ structure representations"
  :export ("*ASSEMBLY-UNIT-LENGTH*"
  "*PRIMITIVE-OBJECTS*"
  "AFTER-BREAKPOINT-TRAP"
+ "*ALLOC-SIGNAL*"
  "ANY-REG-SC-NUMBER" "ARRAY-DATA-SLOT" "ARRAY-DIMENSIONS-OFFSET"
  "ARRAY-DISPLACED-P-SLOT" "ARRAY-DISPLACEMENT-SLOT"
  "ARRAY-ELEMENTS-SLOT" "ARRAY-FILL-POINTER-P-SLOT"

src/code/cold-init.lisp

@@ -287,6 +287,7 @@ UNIX-like systems, UNIX-STATUS is used as the status code."
 
 (defun reinit ()
  (setf *default-external-format* nil)
+ (setf sb!alien::*default-c-string-external-format* nil)
  (without-interrupts
  (without-gcing
  (os-cold-init-or-reinit)

src/code/early-alieneval.lisp

@@ -26,3 +26,5 @@
 ;;; Lisp idiom for C's return type "void" (which is likely
 ;;; why it's set when when translating return values)
 (defvar *values-type-okay* nil)
+
+(defvar *default-c-string-external-format* nil)

src/code/error.lisp

@@ -124,6 +124,24 @@
  stream ':external-format (stream-external-format stream)
  octets)))))
 
+(define-condition c-string-encoding-error (character-encoding-error)
+ ((external-format :initarg :external-format :reader c-string-encoding-error-external-format))
+ (:report
+ (lambda (c s)
+ (format s "~@<c-string encoding error (:external-format ~S): ~2I~_~
+ the character with code ~D cannot be encoded.~@:>"
+ (c-string-encoding-error-external-format c)
+ (character-encoding-error-code c)))))
+
+(define-condition c-string-decoding-error (character-decoding-error)
+ ((external-format :initarg :external-format :reader c-string-decoding-error-external-format))
+ (:report
+ (lambda (c s)
+ (format s "~@<c-string decoding error (:external-format ~S): ~2I~_~
+ the octet sequence ~S cannot be decoded.~@:>"
+ (c-string-decoding-error-external-format c)
+ (character-decoding-error-octets c)))))
+
 (define-condition control-stack-exhausted (storage-condition)
  ()
  (:report

src/code/external-formats/enc-cyr.lisp

@@ -169,11 +169,11 @@
  (let ((koi8-r-byte (code->koi8-r-mapper bits)))
  (if koi8-r-byte
  (setf (sap-ref-8 sap tail) koi8-r-byte)
- (stream-encoding-error-and-handle stream bits)))
+ (external-format-encoding-error stream bits)))
  (let ((code (koi8-r->code-mapper byte)))
  (if code
  (code-char code)
- (stream-decoding-error stream byte)))) ;; TODO -- error check
+ (external-format-decoding-error stream byte)))) ;; TODO -- error check
 
 (define-unibyte-mapper koi8-u->code-mapper code->koi8-u-mapper
  (#x80 #x2500) ; BOX DRAWINGS LIGHT HORIZONTAL
@@ -344,11 +344,11 @@
  (let ((koi8-u-byte (code->koi8-u-mapper bits)))
  (if koi8-u-byte
  (setf (sap-ref-8 sap tail) koi8-u-byte)
- (stream-encoding-error-and-handle stream bits)))
+ (external-format-encoding-error stream bits)))
  (let ((code (koi8-u->code-mapper byte)))
  (if code
  (code-char code)
- (stream-decoding-error stream byte)))) ;; TODO -- error check
+ (external-format-decoding-error stream byte)))) ;; TODO -- error check
 
 (define-unibyte-mapper x-mac-cyrillic->code-mapper code->x-mac-cyrillic-mapper
  (#x80 #x0410) ; CYRILLIC CAPITAL LETTER A
@@ -514,8 +514,8 @@
  (let ((x-mac-cyrillic-byte (code->x-mac-cyrillic-mapper bits)))
  (if x-mac-cyrillic-byte
  (setf (sap-ref-8 sap tail) x-mac-cyrillic-byte)
- (stream-encoding-error-and-handle stream bits)))
+ (external-format-encoding-error stream bits)))
  (let ((code (x-mac-cyrillic->code-mapper byte)))
  (if code
  (code-char code)
- (stream-decoding-error stream byte)))) ;; TODO -- error check
+ (external-format-decoding-error stream byte)))) ;; TODO -- error check