Serendipity Forums

[onli]

I tried to add pubsubhubbub to serendipity. First of all, it didn't work. Maybe anyone sees why it fails?

When publishing an entry, everything seems to work fine, the answer is a 204. But the hub don't find any debug-information to any of my feeds.

My changes: The funtions_entry.inc.php and this publisher, which is modified (without php5-parts, as far as I can tell), and of course changes to the feeds.

Here the diffs to get a better overview:

Code: Select all

diff -Nur serendipity-nightly/include/functions_entries.inc.php functions_entries.inc.txt
--- serendipity-nightly/include/functions_entries.inc.php   2009-11-25 21:41:43.000000000 +0100
+++ functions_entries.inc.txt   2010-01-18 17:09:16.000000000 +0100
@@ -1400,6 +1400,24 @@
         // Now that plugins are executed, we go ahead into the Temple of Doom and send possibly failing trackbacks.
         // First, original list of references is restored (inside the function call)
         serendipity_handle_references($entry['id'], $serendipity['blogTitle'], $drafted_entry['title'], $drafted_entry['body'] . $drafted_entry['extended'], false);
+        include_once 'publisher.php';
+        $hub_url = 'http://pubsubhubbub.appspot.com/';
+        $topic_url = array();
+        $topic_url[] = $serendipity['baseURL'] . serendipity_rewriteURL(PATH_FEEDS .'/index.rss2', 'serendipityHTTPPath');
+        $topic_url[] = $serendipity['baseURL'] . serendipity_rewriteURL(PATH_FEEDS .'/atom10.xml', 'serendipityHTTPPath');
+       
+        foreach($categories as $category) {
+            #entries without a category don't belong to a category-feed
+            if ($category > 0) {
+                $topic_url[] = $serendipity['baseURL'] . serendipity_feedCategoryURL($category, 'serendipityHTTPPath');
+            }
+        }
+        $p = new Publisher($hub_url);
+        if (! $p->publish_update($topic_url)) {
+            echo "Publishing to hub failed!";
+            print_r($p->last_response());
+        }
+       
     }
 
     return (int)$entry['id'];

Code: Select all

cat /var/www/bundled-libs/publisher.php
<?php

// a PHP client library for pubsubhubbub
// as defined at http://code.google.com/p/pubsubhubbub/
// original written by Josh Fraser | joshfraser.com | josh@eventvue.com
// Released under Apache License 2.0

class Publisher {
   
    var $hub_url;
    var $last_response;
   
    // create a new Publisher
    function __construct($hub_url) {
       
        if (!isset($hub_url))
            throw new Exception('Please specify a hub url');
       
        if (!preg_match("|^https?://|i",$hub_url))
            throw new Exception('The specified hub url does not appear to be valid: '.$hub_url);
           
        $this->hub_url = $hub_url;
    }

    // accepts either a single url or an array of urls
    function publish_update($topic_urls, $http_function = false) {
        if (!isset($topic_urls))
            throw new Exception('Please specify a topic url');
       
        // check that we're working with an array
        if (!is_array($topic_urls)) {
            $topic_urls = array($topic_urls);
        }
       
        // check each topic url
        foreach ($topic_urls as $topic_url) {

            // lightweight check that we're actually working w/ a valid url
            if (!preg_match("|^https?://|i",$topic_url))
                throw new Exception('The specified topic url does not appear to be valid: '.$topic_url);
        }

        return $this->http_post($this->hub_url, $topic_urls);
    }

    // returns any error message from the latest request
    function last_response() {
        return $this->last_response;
    }
   
    // default http function that uses curl to post to the hub endpoint
    private function http_post($url, $topic_urls) {
        @include_once 'HTTP/Request.php';
        if (!class_exists('HTTP_Request')) {
            return false;
        }
        $sender = &new HTTP_Request($url);
        $sender->setMethod(HTTP_REQUEST_METHOD_POST);

        $sender->addPostData('hub.mode', 'publish');
        foreach ($topic_urls as $topic_url) {
            $sender->addPostData('hub.url', $topic_url);
        }
        $sender->addHeader('hub', 'application/x-www-form-urlencoded');
       
        $sender->sendRequest();
       
        $this->last_response = $sender->getResponseBody();
        if ($sender->getResponseCode() == 204) {
            return true;
        }
        return false;
    }
}

?>

Code: Select all

diff -Nur serendipity-nightly/templates/default/feed_2.0.tpl /var/www/templates/default/feed_2.0.tpl
--- serendipity-nightly/templates/default/feed_2.0.tpl   2006-07-25 11:42:28.000000000 +0200
+++ /var/www/templates/default/feed_2.0.tpl   2010-01-15 15:03:31.000000000 +0100
@@ -9,6 +9,7 @@
    xmlns:content="http://purl.org/rss/1.0/modules/content/"
    {$namespace_display_dat}>
 <channel>
+    <atom:link rel="hub" href="http://pubsubhubbub.appspot.com/" xmlns:atom="http://www.w3.org/2005/Atom" />
     <title>{$metadata.title}</title>
     <link>{$metadata.link}</link>
     <description>{$metadata.description}</description>


Code: Select all

diff -Nur serendipity-nightly/templates/default/feed_atom1.0.tpl /var/www/templates/default/feed_atom1.0.tpl
--- serendipity-nightly/templates/default/feed_atom1.0.tpl   2008-03-17 15:34:16.000000000 +0100
+++ /var/www/templates/default/feed_atom1.0.tpl   2010-01-15 15:17:51.000000000 +0100
@@ -9,6 +9,7 @@
    xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
    xmlns:wfw="http://wellformedweb.org/CommentAPI/">
     <link href="{$self_url}" rel="self" title="{$metadata.title}" type="application/atom+xml" />
+    <link rel="hub" href="http://pubsubhubbub.appspot.com/" />
     <link href="{$serendipityBaseURL}"                        rel="alternate"    title="{$metadata.title}" type="text/html" />
     <link href="{$serendipityBaseURL}rss.php?version=2.0"     rel="alternate"    title="{$metadata.title}" type="application/rss+xml" />
     <title type="html">{$metadata.title}</title>

Pubsubhubbub is an open protocol from google to achieve faster rss-delivery. Instead of the reader polling the site, the site notifies a hub and the hub notifies the reader.

[garvinhicking]

Hi!

Wouldn't it be better to do this as an event plugin? Then I could also test it better, I'd love to get my hands into this as well.

Maybe pubsubhubbub sends cache headers and thus does not get updated versions of the feed? Try to send it to rss.php?version=2.0&nocache=true ?

Regards,
Garvin

[onli]

I can't do this as a plugin alone, because the feeds have to be manipulated and until know, no event-hooks for that seem to exist. Shall we add them, or did I oversee them? I thought that it's possible that this fits in the core.

The cache don't seem to be the problem (didn't change the result). The problem seems to occur already when submitting the entry.

[garvinhicking]

Hi!

I think the better way would be to patch the rss.php so that additionally to {$namespace_display_dat} we have a {$channel_display_dat}?

The cache don't seem to be the problem (didn't change the result). The problem seems to occur already when submitting the entry.

Hm, sadly I yet have no experience at all about pubsubhub....

Regards,
Garvin

[onli]

Ok. It's still not working completely, but the test-pubhubserver on my system says that POSTs arrived. So maybe tihs just need some subscribers, if not I hope you find the error(s) (and I will keep looking).

Made a plugin out of that, the patchs to the core are attached.

[onli]

And the plugin (attachment-amount is limited to three?)

[garvinhicking]

Hi!

Great, I patched the SVN core.

About the plugin, maybe you could try instead of a rewritten URL to directly point to rss.php?version=2.0&nocache=true -- I don't think that actually a URL like /feeds/index.rss2&nocache=true would work at all, like the category feed URL currently contains...

Regards,
Garvin

[onli]

Sorry Garvin, but this time I don't understand you
Why shouldn't the normal rss-urls work? And what do you mean with "like the category feed URL currently contains..."?

Thanks for patchng the core.

[garvinhicking]

Hi!

Query parameters to index.rss2 URLs might not work, especially if you have /feeds/index.rss&nocache=true, because it would need to be /feeds/index.rss?nocache=true - since URL rewriting is enabled here, it might happen that the parameters are not passed forth as GET to the actual script.

By using rss.php directly, you can make absolutely sure that all parameters come in, no matter if someone uses URL Rewriting or not.

Regards,
Garvin

[onli]

Ah, ok. But I'm unsure: The hub compares the url with the subscribers. Isn't it possible that because the parameter the hub fails to see that this is a request for the same url without the parameter? More of the same problem with using directly the rss.php. And theoretically, the publisher (we) are even proposed to use last_modified-headers and etag. But the attached version contains links to the rss.php, but the more I think about it, the more I think that this can't work... the version above has a better chance to work

As the posts arrive at the moment, I guess that there need to be subscribers - in my testinstallation of a hub, this part didn't work. Would be great if you or someone else could test that.

[garvinhicking]

Hi!

Ah, I didn't know how that worked. But how does pubsubhubbub then handle different URLs? For many blog systems there are distinct URLs, and if they have parameters they would never really matchup as a single canonical URL?

But to test if it works at all, I'd start testing witht the rss.php URL instead of the 'rewritten' URLs?

Regards,
Garvin

[onli]

I don't know how such a system could be handled by a hub. One would have to ignore all parameters and check the feeds anyway.

But yes, testing could be done with the rss.php. We only have to remember to use them to subscribe.

[onli]

Seems like i introduced a bug with the patch to the rss.php. 'channel_dat' also need to be unset from $entries. Otherwise and with current dev-build, a entry containing only "<" gets appended to the rss.

Code: Select all

diff -Nur serendipity-nightly/rss.php /var/www/rss.php
--- serendipity-nightly/rss.php   2010-01-21 10:20:41.000000000 +0100
+++ /var/www/rss.php   2010-10-10 16:25:11.386515729 +0200
@@ -249,6 +249,7 @@
 $namespace_display_dat = $entries['display_dat'];
 $channel_display_dat = $entries['channel_dat'];
 unset($entries['display_dat']);
+unset($entries['channel_dat']);
 
 $serendipity['smarty']->assign_by_ref('metadata', $metadata);
 $serendipity['smarty']->assign_by_ref('entries', $entries);

[garvinhicking]

Hi!

Committed!

Regards,
Garvin

[blog.brockha.us]

Hmm! This looks very interesting. Is there a public plugin already? Didn't find something in my plugin list, perhaps because my S9Y installation is outdated and needs the new hooks for that plugin?