Charset bug in receent beta?

Discussion corner for Developers of Serendipity.
Post Reply
User avatar
Lux
Regular
Posts: 761
Joined: Fri Aug 12, 2005 4:36 pm
Location: Grüt, Zürich, Switzerland
Contact:

Charset bug in receent beta?

Post by Lux » Sat Dec 06, 2014 10:02 am

Hi!

I upgraded my blog to the latest beta yesterday and it ran well for the remaining day.

Now I got a comment on a blog post and from that time on I get the following error message:

Code: Select all

== TESTING ERROR MODE ==

Fatal error: Uncaught exception 'ErrorException' with message 'htmlspecialchars(): charset `LANG_CHARSET' not supported, assuming iso-8859-1' in /srv/www/deimeke.net/dirk/blog/include/compat.inc.php:126
Stack trace:
#0 [internal function]: errorToExceptionHandler(2, 'htmlspecialchar...', '/srv/www/deimek...', 425, Array)
#1 /srv/www/deimeke.net/dirk/blog/include/compat.inc.php(425): htmlspecialchars('DELETE FROM ser...', 2, 'LANG_CHARSET', true)
#2 /srv/www/deimeke.net/dirk/blog/include/db/mysql.inc.php(99): serendipity_specialchars('DELETE FROM ser...')
#3 /srv/www/deimeke.net/dirk/blog/include/functions_config.inc.php(475): serendipity_db_query('DELETE FROM ser...')
#4 /srv/www/deimeke.net/dirk/blog/include/functions_config.inc.php(506): serendipity_issueAutologin(false)
#5 /srv/www/deimeke.net/dirk/blog/include/functions_config.inc.php(427): serendipity_checkAutologin('4164ca482e837e6...', 'B2u2EzDnZhs=')
#6 /srv/www/deimeke.net/dirk/blog/serendipity_config.inc.php(352): serendipity_login(false)
#7 /srv/www/deimeke.net in /srv/www/deimeke.net/dirk/blog/include/compat.inc.php on line 126
Any hints?

Cheers

Dirk

User avatar
Lux
Regular
Posts: 761
Joined: Fri Aug 12, 2005 4:36 pm
Location: Grüt, Zürich, Switzerland
Contact:

Re: Charset bug in receent beta?

Post by Lux » Sat Dec 06, 2014 10:05 am

Hi!

Maybe this commit is the reason. My blog is still running on PHP 5.3

Code: Select all

commit 92afc37753273f5897d13f60185e434fdcbf4d12
Author: onli <onli89@googlemail.com>
Date:   Sun Nov 23 23:41:08 2014 +0100

    Introduce serendipity_specialchars-wrapper for encoding bug (#236)
    PHP 5.4 sets UTF-8 as the default for htmlspecialchars, htmlentities and html_entity_decode. The first two will echo

Cheers

Dirk

User avatar
Timbalu
Regular
Posts: 4598
Joined: Sun May 02, 2004 3:04 pm

Re: Charset bug in receent beta?

Post by Timbalu » Sat Dec 06, 2014 12:24 pm

Hmm very bad. This is a bug.
We will need to find a better place for these wrapper functions, since $charset and LANG_CHARSET aren't defined yet, or have to make compat load them first.

For you with PHP 5.3 and to keep up working, add this to the wrapper function in compat.inc as a workaround until we have fixed the place

Code: Select all

   @define('LANG_CHARSET', 'ISO-8859-1');
right before the returns, eg

Code: Select all

    @define('LANG_CHARSET', 'ISO-8859-1');
    return htmlspecialchars($string, $flags, $encoding, $double_encode);
. You can do that to all three wrappers as a fix workaround..
Regards,
Ian

Serendipity Styx Edition and additional_plugins @ https://ophian.github.io/ @ https://github.com/ophian

User avatar
Lux
Regular
Posts: 761
Joined: Fri Aug 12, 2005 4:36 pm
Location: Grüt, Zürich, Switzerland
Contact:

Re: Charset bug in receent beta?

Post by Lux » Sat Dec 06, 2014 1:28 pm

Timbalu wrote:You can do that to all three wrappers as a fix workaround..
Does not work unfortunately.

Cheers

Dirk

User avatar
Timbalu
Regular
Posts: 4598
Joined: Sun May 02, 2004 3:04 pm

Re: Charset bug in receent beta?

Post by Timbalu » Sat Dec 06, 2014 4:31 pm

Hmm, then you probably have to set this $encoding = LANG_CHARSET to $encoding = 'ISO-8859-1', eg

Code: Select all

function serendipity_specialchars($string, $flags = null, $encoding = 'ISO-8859-1', $double_encode = true) {
and the same in the other wrappers.
Regards,
Ian

Serendipity Styx Edition and additional_plugins @ https://ophian.github.io/ @ https://github.com/ophian

User avatar
Lux
Regular
Posts: 761
Joined: Fri Aug 12, 2005 4:36 pm
Location: Grüt, Zürich, Switzerland
Contact:

Re: Charset bug in receent beta?

Post by Lux » Sat Dec 06, 2014 5:28 pm

Timbalu wrote:Hmm, then you probably have to set this $encoding = LANG_CHARSET to $encoding = 'ISO-8859-1', eg

Code: Select all

function serendipity_specialchars($string, $flags = null, $encoding = 'ISO-8859-1', $double_encode = true) {
and the same in the other wrappers.
We are one step closer.

I am able to see the blog, but the German umlauts are coded wrongly.

Take a look: http://www.deimeke.net/dirk/blog/

Cheers

Dirk

User avatar
Timbalu
Regular
Posts: 4598
Joined: Sun May 02, 2004 3:04 pm

Re: Charset bug in receent beta?

Post by Timbalu » Sat Dec 06, 2014 5:38 pm

Lux wrote:Take a look: http://www.deimeke.net/dirk/blog/
Looks good to me for the non-ascii chars. Might need some clicks to change overall.
It could also very well be, you are using UTF-8 already. In this case you would need to force encoding to $encoding = 'UTF-8'.
Regards,
Ian

Serendipity Styx Edition and additional_plugins @ https://ophian.github.io/ @ https://github.com/ophian

User avatar
Lux
Regular
Posts: 761
Joined: Fri Aug 12, 2005 4:36 pm
Location: Grüt, Zürich, Switzerland
Contact:

Re: Charset bug in receent beta?

Post by Lux » Sat Dec 06, 2014 5:56 pm

Timbalu wrote:
Lux wrote:Take a look: http://www.deimeke.net/dirk/blog/
Looks good to me for the non-ascii chars. Might need some clicks to change overall.
It could also very well be, you are using UTF-8 already. In this case you would need to force encoding to $encoding = 'UTF-8'.
I came to the same idea.

This is what I changed:

Code: Select all

...
function serendipity_specialchars($string, $flags = null, $encoding = 'UTF-8', $double_encode = true) {
...
    @define('LANG_CHARSET', 'UTF-8');
    return htmlspecialchars($string, $flags, $encoding, $double_encode);
...
function serendipity_entities($string, $flags = null, $encoding = 'UTF-8', $double_encode = true) {
...
    @define('LANG_CHARSET', 'UTF-8');
    return htmlentities($string, $flags, $encoding, $double_encode);
...
function serendipity_entity_decode($string, $flags = null, $encoding = 'UTF-8', $double_encode = true) {
...
    @define('LANG_CHARSET', 'UTF-8');
    return html_entity_decode($string, $flags, $encoding, $double_encode);
...
And it looks perfectly well now.

Cheers

Dirk

User avatar
Timbalu
Regular
Posts: 4598
Joined: Sun May 02, 2004 3:04 pm

Re: Charset bug in receent beta?

Post by Timbalu » Sat Dec 06, 2014 6:10 pm

You can now please remove the inserted @define('LANG_CHARSET', 'UTF-8'); defines again.
And remember, this is just a dirty workaround until we have fixed the real issue!
Regards,
Ian

Serendipity Styx Edition and additional_plugins @ https://ophian.github.io/ @ https://github.com/ophian

User avatar
Lux
Regular
Posts: 761
Joined: Fri Aug 12, 2005 4:36 pm
Location: Grüt, Zürich, Switzerland
Contact:

Re: Charset bug in receent beta?

Post by Lux » Sat Dec 06, 2014 6:27 pm

Timbalu wrote:You can now please remove the inserted @define('LANG_CHARSET', 'UTF-8'); defines again.
And remember, this is just a dirty workaround until we have fixed the real issue!
https://github.com/s9y/Serendipity/issues/246

Thanks Ian!

Cheers

Dirk

User avatar
Timbalu
Regular
Posts: 4598
Joined: Sun May 02, 2004 3:04 pm

Re: Charset bug in receent beta?

Post by Timbalu » Sat Dec 06, 2014 6:35 pm

Oh well, I had already reopened the original issue for this
https://github.com/s9y/Serendipity/issues/236
Regards,
Ian

Serendipity Styx Edition and additional_plugins @ https://ophian.github.io/ @ https://github.com/ophian

User avatar
Lux
Regular
Posts: 761
Joined: Fri Aug 12, 2005 4:36 pm
Location: Grüt, Zürich, Switzerland
Contact:

Re: Charset bug in receent beta?

Post by Lux » Sat Dec 06, 2014 6:51 pm

Timbalu wrote:Oh well, I had already reopened the original issue for this
https://github.com/s9y/Serendipity/issues/236
Ok, I closed mine.

Cheers

Dirk

User avatar
onli
Regular
Posts: 2241
Joined: Tue Sep 09, 2008 10:04 pm
Contact:

Re: Charset bug in receent beta?

Post by onli » Sun Dec 07, 2014 6:53 pm

Hi Dirk
Thanks for finding this before the release!

I just pushed a possible fix into the 2.0-branch: https://github.com/s9y/Serendipity/comm ... 15740830bf

Could you test this on your blog? The only changes are in the include/compat.inc.php, and the error/exception should disappear, while hopefully only some debug messages might appear garbled.

User avatar
Lux
Regular
Posts: 761
Joined: Fri Aug 12, 2005 4:36 pm
Location: Grüt, Zürich, Switzerland
Contact:

Re: Charset bug in receent beta?

Post by Lux » Sun Dec 07, 2014 7:00 pm

onli wrote:Could you test this on your blog? The only changes are in the include/compat.inc.php, and the error/exception should disappear, while hopefully only some debug messages might appear garbled.
Many thanks, Malte.

It works for me!

Cheers

Dirk

User avatar
onli
Regular
Posts: 2241
Joined: Tue Sep 09, 2008 10:04 pm
Contact:

Re: Charset bug in receent beta?

Post by onli » Sun Dec 07, 2014 7:04 pm

Thanks!

Please have a lookout in the following days, if strange encoding issues appear - comments, trackbacks, stuff that is somehow tangential to the normal workflow - this patch might be the culprit.

Post Reply