Page 1 of 1

"noindex" for nearly all pages

Posted: Tue Jul 31, 2018 1:50 am
by stephanbrunker
This is the problem from the german subforum: viewtopic.php?f=10&p=10450914&sid=a202a ... #p10450914 and i was able to hack it. Because I don't know if what I've done is right and I think this is a bug to fix, I repost it here.

Basically, all my pages in my blog weren't indexed by Google because of the "noindex" header. Only the static pages are indexed. I followed the problem through the source and one piece of the problem are the these lines in the index.tpl (I use the 2k11 theme):

Code: Select all

{if ($view == "entry" || $view == "start" || $view == "feed" || $view == "plugin" || $staticpage_pagetitle != "" || $robots_index == 'index')}
    <meta name="robots" content="index,follow">
    <meta name="robots" content="noindex,follow">
That means that either the $view is not one of possibilies for index and/or $view is not defined at all. The latter could be true, because in the transfer between the $serendipity[] array to the $serenditipy['smarty']->assign ( on line 1062) the parameter 'view' is missing:

Code: Select all

function serendipity_smarty_init($vars = array()) {
                'head_charset'              => LANG_CHARSET,
                'head_version'              => $serendipity['version'],
                'head_title'                => $serendipity['head_title'],
                'head_subtitle'             => $serendipity['head_subtitle'],
                'head_link_stylesheet'      => $serendipity['smarty_vars']['head_link_stylesheet'],
                'head_link_script'          => $serendipity['smarty_vars']['head_link_script'],
                'head_link_stylesheet_frontend' => $serendipity['smarty_vars']['head_link_stylesheet_frontend'],

                'is_xhtml'                  => true,
                'use_popups'                => $serendipity['enablePopup'],
                'use_backendpopups'         => $serendipity['enableBackendPopup'],
                'force_backendpopups'       => $force_backendpopups,
                'is_embedded'               => (!$serendipity['embed'] || $serendipity['embed'] === 'false' || $serendipity['embed'] === false) ? false : true,
                'is_raw_mode'               => $serendipity['smarty_raw_mode'],
                'is_logged_in'              => serendipity_userLoggedIn(),

                'entry_id'                  => (isset($serendipity['GET']['id']) && is_numeric($serendipity['GET']['id'])) ? $serendipity['GET']['id'] : false,
                'is_single_entry'           => (isset($serendipity['GET']['id']) && is_numeric($serendipity['GET']['id'])),

                'blogTitle'                 => $serendipity['blogTitle'],
                'blogSubTitle'              => (!empty($serendipity['blogSubTitle']) ? $serendipity['blogSubTitle'] : ''),
                'blogDescription'           => $serendipity['blogDescription'],

                'serendipityHTTPPath'       => $serendipity['serendipityHTTPPath'],
                'serendipityDefaultBaseURL' => $serendipity['defaultBaseURL'],
                'serendipityBaseURL'        => $serendipity['baseURL'],
                'serendipityRewritePrefix'  => $serendipity['rewrite'] == 'none' ? $serendipity['indexFile'] . '?/' : '',
                'serendipityIndexFile'      => $serendipity['indexFile'],
                'serendipityVersion'        => ($serendipity['expose_s9y'] ? $serendipity['version'] : ''),

                'view'                      => $serendipity['view'],
                'lang'                      => $serendipity['lang'],
                'category'                  => $category,
                'category_info'             => $category_info,
                'template'                  => $serendipity['template'],
                'template_backend'          => $serendipity['template_backend'],
                'wysiwygToolbar'            => $serendipity['wysiwygToolbar'],
                'wysiwyg_customPlugin'      => $wysiwyg_customPlugin,
                'wysiwyg_customConfig'      => $wysiwyg_customConfig,
                'use_autosave'              => (serendipity_db_bool($serendipity['use_autosave']) ? 'true' : 'false'),

                'dateRange'                 => (!empty($serendipity['range']) ? $serendipity['range'] : array())

After inserting there the line

Code: Select all

                'view'                      => $serendipity['view'],
the start page changed to 'index,follow'. But not the entries linked on the start page. Because one of the values for 'view' is 'archives' and looked attractive i tried to add '|| $view == "archives" ' in the index.tpl (2k11) line 12:

Code: Select all

{if ($view == "entry" || $view == "start" || $view == "archives" || $view == "feed" || $view == "plugin" || $staticpage_pagetitle != "" || $robots_index == 'index')}
    <meta name="robots" content="index,follow">
    <meta name="robots" content="noindex,follow">
and then all the full entries changed to 'index,follow' too. Now I could init a new crawling by Google and hopefully, all the 132 skipped pages are now going to be indexed.

I don't know what the origin of this bug is - I simply cannot be true that all pages powered by s9y since forever weren't ever indexed by Google - but I think it important to fix asap.

For the plugins, I only use serendipity_event_multilingual and I don't think that this one is guilty ...

Re: "noindex" for nearly all pages

Posted: Tue Jul 31, 2018 9:54 am
by onli
Thanks for the report, but we have to look a bit further into this. This function definitely works in my blog (, and also in some other s9y blogs I know ( for example).

It is correct that the view variable is not set in, but that can also just mean it is set somewhere else. It is not necessarily set as $view, it can also be part of an array or directly given to smarty... Got it, it is set in, see ... nc.php#L17:

Code: Select all

$uri_addData = array(
    'startpage' => false,
    'uriargs'   => implode('/', serendipity_getUriArguments($uri, true)),
    'view'      => $serendipity['view'],
    'viewtype'  => isset($serendipity['viewtype']) ? $serendipity['viewtype'] : ''
This is then given to smarty in ... .php#L1108.

There are some things that could go wrong here: What is actually given to smarty_init is $serendipity['plugindata']['smartyvars'], and the genpage hook event is executed before, you could have a plugin installed that manipulates that variable. It is also possible smarty_init aborts before setting the variable, one reason is that in your case smarty might be already initialized. That could also happen because of a plugin listening to genpage and helpfully initializing smarty for you.

So, this is not a clear-cut bug in the core. But we should improve the reliability of this by at least making sure that $serendipity['plugindata']['smartyvars'] is always given to smarty, also if smarty is already initialized.

Could you test that for me? Remove your addition to smarty_init for now. Then, in, replace line 27

Code: Select all


Code: Select all

if (count($serendipity['plugindata']['smartyvars']) > 0) {
Now that should work, and if not we should be able to pinpoint it to a plugin listening to the genpage event.

Re: "noindex" for nearly all pages

Posted: Tue Jul 31, 2018 8:28 pm
by stephanbrunker
Hello Onli,

your solution does indeed work. I did just some poking in the pond without understanding what I did. The question is - if it is an issue anymore - what plugin caused this.

As Sidebar plugins I use:

and event plugins:

Re: "noindex" for nearly all pages

Posted: Tue Jul 31, 2018 8:34 pm
by onli
Grep for 'genpage', that's the name of the hook. But actually many plugins catch that hook. customarchives is a candidate, whatever happens it will call smarty_init, and if my theory is right that's enough to trigger this bug. But I'd need to debug it more to be certain. For now I'd just push the patch you tested, thanks for that!