Charset bug in receent beta?

Discussion corner for Developers of Serendipity.
Post Reply
Lux
Regular
Posts: 764
Joined: Fri Aug 12, 2005 4:36 pm
Location: Grüt, Zürich, Switzerland
Contact:

Charset bug in receent beta?

Post by Lux »

Hi!

I upgraded my blog to the latest beta yesterday and it ran well for the remaining day.

Now I got a comment on a blog post and from that time on I get the following error message:

Code: Select all

== TESTING ERROR MODE ==

Fatal error: Uncaught exception 'ErrorException' with message 'htmlspecialchars(): charset `LANG_CHARSET' not supported, assuming iso-8859-1' in /srv/www/deimeke.net/dirk/blog/include/compat.inc.php:126
Stack trace:
#0 [internal function]: errorToExceptionHandler(2, 'htmlspecialchar...', '/srv/www/deimek...', 425, Array)
#1 /srv/www/deimeke.net/dirk/blog/include/compat.inc.php(425): htmlspecialchars('DELETE FROM ser...', 2, 'LANG_CHARSET', true)
#2 /srv/www/deimeke.net/dirk/blog/include/db/mysql.inc.php(99): serendipity_specialchars('DELETE FROM ser...')
#3 /srv/www/deimeke.net/dirk/blog/include/functions_config.inc.php(475): serendipity_db_query('DELETE FROM ser...')
#4 /srv/www/deimeke.net/dirk/blog/include/functions_config.inc.php(506): serendipity_issueAutologin(false)
#5 /srv/www/deimeke.net/dirk/blog/include/functions_config.inc.php(427): serendipity_checkAutologin('4164ca482e837e6...', 'B2u2EzDnZhs=')
#6 /srv/www/deimeke.net/dirk/blog/serendipity_config.inc.php(352): serendipity_login(false)
#7 /srv/www/deimeke.net in /srv/www/deimeke.net/dirk/blog/include/compat.inc.php on line 126
Any hints?

Cheers

Dirk
Lux
Regular
Posts: 764
Joined: Fri Aug 12, 2005 4:36 pm
Location: Grüt, Zürich, Switzerland
Contact:

Re: Charset bug in receent beta?

Post by Lux »

Hi!

Maybe this commit is the reason. My blog is still running on PHP 5.3

Code: Select all

commit 92afc37753273f5897d13f60185e434fdcbf4d12
Author: onli <onli89@googlemail.com>
Date:   Sun Nov 23 23:41:08 2014 +0100

    Introduce serendipity_specialchars-wrapper for encoding bug (#236)
    PHP 5.4 sets UTF-8 as the default for htmlspecialchars, htmlentities and html_entity_decode. The first two will echo

Cheers

Dirk
Timbalu
Regular
Posts: 4598
Joined: Sun May 02, 2004 3:04 pm

Re: Charset bug in receent beta?

Post by Timbalu »

Hmm very bad. This is a bug.
We will need to find a better place for these wrapper functions, since $charset and LANG_CHARSET aren't defined yet, or have to make compat load them first.

For you with PHP 5.3 and to keep up working, add this to the wrapper function in compat.inc as a workaround until we have fixed the place

Code: Select all

   @define('LANG_CHARSET', 'ISO-8859-1');
right before the returns, eg

Code: Select all

    @define('LANG_CHARSET', 'ISO-8859-1');
    return htmlspecialchars($string, $flags, $encoding, $double_encode);
. You can do that to all three wrappers as a fix workaround..
Regards,
Ian

Serendipity Styx Edition and additional_plugins @ https://ophian.github.io/ @ https://github.com/ophian
Lux
Regular
Posts: 764
Joined: Fri Aug 12, 2005 4:36 pm
Location: Grüt, Zürich, Switzerland
Contact:

Re: Charset bug in receent beta?

Post by Lux »

Timbalu wrote:You can do that to all three wrappers as a fix workaround..
Does not work unfortunately.

Cheers

Dirk
Timbalu
Regular
Posts: 4598
Joined: Sun May 02, 2004 3:04 pm

Re: Charset bug in receent beta?

Post by Timbalu »

Hmm, then you probably have to set this $encoding = LANG_CHARSET to $encoding = 'ISO-8859-1', eg

Code: Select all

function serendipity_specialchars($string, $flags = null, $encoding = 'ISO-8859-1', $double_encode = true) {
and the same in the other wrappers.
Regards,
Ian

Serendipity Styx Edition and additional_plugins @ https://ophian.github.io/ @ https://github.com/ophian
Lux
Regular
Posts: 764
Joined: Fri Aug 12, 2005 4:36 pm
Location: Grüt, Zürich, Switzerland
Contact:

Re: Charset bug in receent beta?

Post by Lux »

Timbalu wrote:Hmm, then you probably have to set this $encoding = LANG_CHARSET to $encoding = 'ISO-8859-1', eg

Code: Select all

function serendipity_specialchars($string, $flags = null, $encoding = 'ISO-8859-1', $double_encode = true) {
and the same in the other wrappers.
We are one step closer.

I am able to see the blog, but the German umlauts are coded wrongly.

Take a look: http://www.deimeke.net/dirk/blog/

Cheers

Dirk
Timbalu
Regular
Posts: 4598
Joined: Sun May 02, 2004 3:04 pm

Re: Charset bug in receent beta?

Post by Timbalu »

Lux wrote:Take a look: http://www.deimeke.net/dirk/blog/
Looks good to me for the non-ascii chars. Might need some clicks to change overall.
It could also very well be, you are using UTF-8 already. In this case you would need to force encoding to $encoding = 'UTF-8'.
Regards,
Ian

Serendipity Styx Edition and additional_plugins @ https://ophian.github.io/ @ https://github.com/ophian
Lux
Regular
Posts: 764
Joined: Fri Aug 12, 2005 4:36 pm
Location: Grüt, Zürich, Switzerland
Contact:

Re: Charset bug in receent beta?

Post by Lux »

Timbalu wrote:
Lux wrote:Take a look: http://www.deimeke.net/dirk/blog/
Looks good to me for the non-ascii chars. Might need some clicks to change overall.
It could also very well be, you are using UTF-8 already. In this case you would need to force encoding to $encoding = 'UTF-8'.
I came to the same idea.

This is what I changed:

Code: Select all

...
function serendipity_specialchars($string, $flags = null, $encoding = 'UTF-8', $double_encode = true) {
...
    @define('LANG_CHARSET', 'UTF-8');
    return htmlspecialchars($string, $flags, $encoding, $double_encode);
...
function serendipity_entities($string, $flags = null, $encoding = 'UTF-8', $double_encode = true) {
...
    @define('LANG_CHARSET', 'UTF-8');
    return htmlentities($string, $flags, $encoding, $double_encode);
...
function serendipity_entity_decode($string, $flags = null, $encoding = 'UTF-8', $double_encode = true) {
...
    @define('LANG_CHARSET', 'UTF-8');
    return html_entity_decode($string, $flags, $encoding, $double_encode);
...
And it looks perfectly well now.

Cheers

Dirk
Timbalu
Regular
Posts: 4598
Joined: Sun May 02, 2004 3:04 pm

Re: Charset bug in receent beta?

Post by Timbalu »

You can now please remove the inserted @define('LANG_CHARSET', 'UTF-8'); defines again.
And remember, this is just a dirty workaround until we have fixed the real issue!
Regards,
Ian

Serendipity Styx Edition and additional_plugins @ https://ophian.github.io/ @ https://github.com/ophian
Lux
Regular
Posts: 764
Joined: Fri Aug 12, 2005 4:36 pm
Location: Grüt, Zürich, Switzerland
Contact:

Re: Charset bug in receent beta?

Post by Lux »

Timbalu wrote:You can now please remove the inserted @define('LANG_CHARSET', 'UTF-8'); defines again.
And remember, this is just a dirty workaround until we have fixed the real issue!
https://github.com/s9y/Serendipity/issues/246

Thanks Ian!

Cheers

Dirk
Timbalu
Regular
Posts: 4598
Joined: Sun May 02, 2004 3:04 pm

Re: Charset bug in receent beta?

Post by Timbalu »

Oh well, I had already reopened the original issue for this
https://github.com/s9y/Serendipity/issues/236
Regards,
Ian

Serendipity Styx Edition and additional_plugins @ https://ophian.github.io/ @ https://github.com/ophian
Lux
Regular
Posts: 764
Joined: Fri Aug 12, 2005 4:36 pm
Location: Grüt, Zürich, Switzerland
Contact:

Re: Charset bug in receent beta?

Post by Lux »

Timbalu wrote:Oh well, I had already reopened the original issue for this
https://github.com/s9y/Serendipity/issues/236
Ok, I closed mine.

Cheers

Dirk
onli
Regular
Posts: 2825
Joined: Tue Sep 09, 2008 10:04 pm
Contact:

Re: Charset bug in receent beta?

Post by onli »

Hi Dirk
Thanks for finding this before the release!

I just pushed a possible fix into the 2.0-branch: https://github.com/s9y/Serendipity/comm ... 15740830bf

Could you test this on your blog? The only changes are in the include/compat.inc.php, and the error/exception should disappear, while hopefully only some debug messages might appear garbled.
Lux
Regular
Posts: 764
Joined: Fri Aug 12, 2005 4:36 pm
Location: Grüt, Zürich, Switzerland
Contact:

Re: Charset bug in receent beta?

Post by Lux »

onli wrote:Could you test this on your blog? The only changes are in the include/compat.inc.php, and the error/exception should disappear, while hopefully only some debug messages might appear garbled.
Many thanks, Malte.

It works for me!

Cheers

Dirk
onli
Regular
Posts: 2825
Joined: Tue Sep 09, 2008 10:04 pm
Contact:

Re: Charset bug in receent beta?

Post by onli »

Thanks!

Please have a lookout in the following days, if strange encoding issues appear - comments, trackbacks, stuff that is somehow tangential to the normal workflow - this patch might be the culprit.
Post Reply