Is there a way to keep json_encode()
from returning null
for a string that contains an invalid (non-UTF-8) character?
It can be a pain in the ass to debug in a complex system. It would be much more fitting to actually see the invalid character, or at least have it omitted. As it stands, json_encode()
will silently drop the entire string.
Example (in UTF-8):
$string =
array(utf8_decode("Düsseldorf"), // Deliberately produce broken string
"Washington",
"Nairobi");
print_r(json_encode($string));
Results in
[null,"Washington","Nairobi"]
Desired result:
["D?sseldorf","Washington","Nairobi"]
Note: I am not looking to make broken strings work in json_encode(). I am looking for ways to make it easier to diagnose encoding errors. A null
string isn't helpful for that.
php does try to spew an error, but only if you turn display_errors off. This is odd because the
display_errors
setting is only meant to control whether or not errors are printed to standard output, not whether or not an error is triggered. I want to emphasize that when you havedisplay_errors
on, even though you may see all kinds of other php errors, php doesn't just hide this error, it will not even trigger it. That means it will not show up in any error logs, nor will any custom error_handlers get called. The error just never occurs.Here's some code that demonstrates this:
That bizarre and unfortunate behavior is related to this bug https://bugs.php.net/bug.php?id=47494 and a few others, and doesn't look like it will ever be fixed.
workaround:
Cleaning the string before passing it to json_encode may be a workable solution.
http://php.net/manual/en/function.iconv.php
The manual says
So by first removing the problematic characters, in theory json_encode() shouldnt get anything it will choke on and fail with. I haven't verified that the output of iconv with the
//IGNORE
flag is perfectly compatible with json_encodes notion of what valid utf8 characters are, so buyer beware...as there may be edge cases where it still fails. ugh, I hate character set issues.Edit
in php 7.2+, there seems to be some new flags for
json_encode
:JSON_INVALID_UTF8_IGNORE
andJSON_INVALID_UTF8_SUBSTITUTE
There's not much documentation yet, but for now, this test should help you understand expected behavior: https://github.com/php/php-src/blob/master/ext/json/tests/json_encode_invalid_utf8.phpt
And, in php 7.3+ there's the new flag
JSON_THROW_ON_ERROR
. See http://php.net/manual/en/class.jsonexception.php