U+FFFD is the input substitution character

* libguile/ports.c (UNICODE_REPLACEMENT_CHARACTER):
* libguile/ports.c (peek_utf8_codepoint)
  (scm_port_decode_char, peek_iconv_codepoint):
* module/ice-9/sports.scm (peek-char-and-len/utf8):
  (peek-char-and-len/iconv): Return U+FFFD when we get a decoding error
  when reading, instead of '?', in accordance with Unicode
  recommendations.
* test-suite/tests/iconv.test:
* test-suite/tests/ports.test:
* test-suite/tests/rdelim.test: Update tests.
* NEWS: Update.

1 files changed, 7 insertions, 0 deletions

diff --git a/NEWS b/NEWS
index 7165f8d9e..3e64129e4 100644
--- a/NEWS
+++ b/NEWS
@@ -71,6 +71,13 @@ raise an error on bad input.  Guile now raises an error without
 advancing the read pointer.  To skip over a bad encoding, set the port
 conversion strategy to "substitute" and read a substitute character.
 
+** Decoding errors with `substitute' strategy return U+FFFD
+
+It used to be that decoding errors with the `substitute' conversion
+strategy would replace the bad bytes with a `?' character.  This has
+been changed to use the standard U+FFFD REPLACEMENT CHARACTER, in
+accordance with the Unicode recommendations.
+
 ** API to define new port types from C has changed
 
 See the newly expanded "I/O Extensions" in the manual, for full details.