pubsubhubbub

Discussion corner for Developers of Serendipity.
onli
Regular
Posts: 2830
Joined: Tue Sep 09, 2008 10:04 pm
Contact:

pubsubhubbub

Post by onli »

I tried to add pubsubhubbub to serendipity. First of all, it didn't work. Maybe anyone sees why it fails?

When publishing an entry, everything seems to work fine, the answer is a 204. But the hub don't find any debug-information to any of my feeds.

My changes: The funtions_entry.inc.php and this publisher, which is modified (without php5-parts, as far as I can tell), and of course changes to the feeds.

Here the diffs to get a better overview:

Code: Select all

diff -Nur serendipity-nightly/include/functions_entries.inc.php functions_entries.inc.txt 
--- serendipity-nightly/include/functions_entries.inc.php	2009-11-25 21:41:43.000000000 +0100
+++ functions_entries.inc.txt	2010-01-18 17:09:16.000000000 +0100
@@ -1400,6 +1400,24 @@
         // Now that plugins are executed, we go ahead into the Temple of Doom and send possibly failing trackbacks.
         // First, original list of references is restored (inside the function call)
         serendipity_handle_references($entry['id'], $serendipity['blogTitle'], $drafted_entry['title'], $drafted_entry['body'] . $drafted_entry['extended'], false);
+        include_once 'publisher.php';
+        $hub_url = 'http://pubsubhubbub.appspot.com/';
+        $topic_url = array();
+        $topic_url[] = $serendipity['baseURL'] . serendipity_rewriteURL(PATH_FEEDS .'/index.rss2', 'serendipityHTTPPath');
+        $topic_url[] = $serendipity['baseURL'] . serendipity_rewriteURL(PATH_FEEDS .'/atom10.xml', 'serendipityHTTPPath');
+        
+        foreach($categories as $category) {
+            #entries without a category don't belong to a category-feed
+            if ($category > 0) {
+                $topic_url[] = $serendipity['baseURL'] . serendipity_feedCategoryURL($category, 'serendipityHTTPPath');
+            }
+        }
+        $p = new Publisher($hub_url);
+        if (! $p->publish_update($topic_url)) {
+            echo "Publishing to hub failed!";
+            print_r($p->last_response());
+        }
+        
     }
 
     return (int)$entry['id'];

Code: Select all

cat /var/www/bundled-libs/publisher.php 
<?php

// a PHP client library for pubsubhubbub
// as defined at http://code.google.com/p/pubsubhubbub/
// original written by Josh Fraser | joshfraser.com | josh@eventvue.com
// Released under Apache License 2.0

class Publisher {
    
    var $hub_url;
    var $last_response;
    
    // create a new Publisher
    function __construct($hub_url) {
        
        if (!isset($hub_url))
            throw new Exception('Please specify a hub url');
        
        if (!preg_match("|^https?://|i",$hub_url)) 
            throw new Exception('The specified hub url does not appear to be valid: '.$hub_url);
            
        $this->hub_url = $hub_url;
    }

    // accepts either a single url or an array of urls
    function publish_update($topic_urls, $http_function = false) {
        if (!isset($topic_urls))
            throw new Exception('Please specify a topic url');
        
        // check that we're working with an array
        if (!is_array($topic_urls)) {
            $topic_urls = array($topic_urls);
        }
        
        // check each topic url 
        foreach ($topic_urls as $topic_url) {

            // lightweight check that we're actually working w/ a valid url
            if (!preg_match("|^https?://|i",$topic_url)) 
                throw new Exception('The specified topic url does not appear to be valid: '.$topic_url);
        }

        return $this->http_post($this->hub_url, $topic_urls);
    }

    // returns any error message from the latest request
    function last_response() {
        return $this->last_response;
    }
    
    // default http function that uses curl to post to the hub endpoint
    private function http_post($url, $topic_urls) {
        @include_once 'HTTP/Request.php';
        if (!class_exists('HTTP_Request')) {
            return false;
        }
        $sender = &new HTTP_Request($url);
        $sender->setMethod(HTTP_REQUEST_METHOD_POST);

        $sender->addPostData('hub.mode', 'publish');
        foreach ($topic_urls as $topic_url) {
            $sender->addPostData('hub.url', $topic_url);
        }
        $sender->addHeader('hub', 'application/x-www-form-urlencoded');
        
        $sender->sendRequest();
        
        $this->last_response = $sender->getResponseBody();
        if ($sender->getResponseCode() == 204) {
            return true;
        }
        return false;
    }
}

?>

Code: Select all

diff -Nur serendipity-nightly/templates/default/feed_2.0.tpl /var/www/templates/default/feed_2.0.tpl 
--- serendipity-nightly/templates/default/feed_2.0.tpl	2006-07-25 11:42:28.000000000 +0200
+++ /var/www/templates/default/feed_2.0.tpl	2010-01-15 15:03:31.000000000 +0100
@@ -9,6 +9,7 @@
    xmlns:content="http://purl.org/rss/1.0/modules/content/"
    {$namespace_display_dat}>
 <channel>
+    <atom:link rel="hub" href="http://pubsubhubbub.appspot.com/" xmlns:atom="http://www.w3.org/2005/Atom" />
     <title>{$metadata.title}</title>
     <link>{$metadata.link}</link>
     <description>{$metadata.description}</description>

Code: Select all

diff -Nur serendipity-nightly/templates/default/feed_atom1.0.tpl /var/www/templates/default/feed_atom1.0.tpl 
--- serendipity-nightly/templates/default/feed_atom1.0.tpl	2008-03-17 15:34:16.000000000 +0100
+++ /var/www/templates/default/feed_atom1.0.tpl	2010-01-15 15:17:51.000000000 +0100
@@ -9,6 +9,7 @@
    xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
    xmlns:wfw="http://wellformedweb.org/CommentAPI/">
     <link href="{$self_url}" rel="self" title="{$metadata.title}" type="application/atom+xml" />
+    <link rel="hub" href="http://pubsubhubbub.appspot.com/" />
     <link href="{$serendipityBaseURL}"                        rel="alternate"    title="{$metadata.title}" type="text/html" />
     <link href="{$serendipityBaseURL}rss.php?version=2.0"     rel="alternate"    title="{$metadata.title}" type="application/rss+xml" />
     <title type="html">{$metadata.title}</title>
Pubsubhubbub is an open protocol from google to achieve faster rss-delivery. Instead of the reader polling the site, the site notifies a hub and the hub notifies the reader.
Attachments

[The extension tpl has been deactivated and can no longer be displayed.]

[The extension tpl has been deactivated and can no longer be displayed.]

[The extension php has been deactivated and can no longer be displayed.]

garvinhicking
Core Developer
Posts: 30022
Joined: Tue Sep 16, 2003 9:45 pm
Location: Cologne, Germany
Contact:

Re: pubsubhubbub

Post by garvinhicking »

Hi!

Wouldn't it be better to do this as an event plugin? Then I could also test it better, I'd love to get my hands into this as well.

Maybe pubsubhubbub sends cache headers and thus does not get updated versions of the feed? Try to send it to rss.php?version=2.0&nocache=true ?

Regards,
Garvin
# Garvin Hicking (s9y Developer)
# Did I help you? Consider making me happy: http://wishes.garv.in/
# or use my PayPal account "paypal {at} supergarv (dot) de"
# My "other" hobby: http://flickr.garv.in/
onli
Regular
Posts: 2830
Joined: Tue Sep 09, 2008 10:04 pm
Contact:

Re: pubsubhubbub

Post by onli »

I can't do this as a plugin alone, because the feeds have to be manipulated and until know, no event-hooks for that seem to exist. Shall we add them, or did I oversee them? I thought that it's possible that this fits in the core.

The cache don't seem to be the problem (didn't change the result). The problem seems to occur already when submitting the entry.
garvinhicking
Core Developer
Posts: 30022
Joined: Tue Sep 16, 2003 9:45 pm
Location: Cologne, Germany
Contact:

Re: pubsubhubbub

Post by garvinhicking »

Hi!

I think the better way would be to patch the rss.php so that additionally to {$namespace_display_dat} we have a {$channel_display_dat}?
The cache don't seem to be the problem (didn't change the result). The problem seems to occur already when submitting the entry.
Hm, sadly I yet have no experience at all about pubsubhub....

Regards,
Garvin
# Garvin Hicking (s9y Developer)
# Did I help you? Consider making me happy: http://wishes.garv.in/
# or use my PayPal account "paypal {at} supergarv (dot) de"
# My "other" hobby: http://flickr.garv.in/
onli
Regular
Posts: 2830
Joined: Tue Sep 09, 2008 10:04 pm
Contact:

Re: pubsubhubbub

Post by onli »

Ok. It's still not working completely, but the test-pubhubserver on my system says that POSTs arrived. So maybe tihs just need some subscribers, if not I hope you find the error(s) (and I will keep looking).

Made a plugin out of that, the patchs to the core are attached.
Attachments
feed_atom1.0.tpl.diff.txt
(739 Bytes) Downloaded 426 times
feed_2.0.tpl.diff.txt
(446 Bytes) Downloaded 423 times
rss.php.diff.txt
(760 Bytes) Downloaded 455 times
onli
Regular
Posts: 2830
Joined: Tue Sep 09, 2008 10:04 pm
Contact:

Re: pubsubhubbub

Post by onli »

And the plugin (attachment-amount is limited to three?)
Attachments
serendipity_event_pubsubhubbub-0.1.tar.gz
(2.5 KiB) Downloaded 455 times
garvinhicking
Core Developer
Posts: 30022
Joined: Tue Sep 16, 2003 9:45 pm
Location: Cologne, Germany
Contact:

Re: pubsubhubbub

Post by garvinhicking »

Hi!

Great, I patched the SVN core.

About the plugin, maybe you could try instead of a rewritten URL to directly point to rss.php?version=2.0&nocache=true -- I don't think that actually a URL like /feeds/index.rss2&nocache=true would work at all, like the category feed URL currently contains...

Regards,
Garvin
# Garvin Hicking (s9y Developer)
# Did I help you? Consider making me happy: http://wishes.garv.in/
# or use my PayPal account "paypal {at} supergarv (dot) de"
# My "other" hobby: http://flickr.garv.in/
onli
Regular
Posts: 2830
Joined: Tue Sep 09, 2008 10:04 pm
Contact:

Re: pubsubhubbub

Post by onli »

Sorry Garvin, but this time I don't understand you :)
Why shouldn't the normal rss-urls work? And what do you mean with "like the category feed URL currently contains..."?

Thanks for patchng the core.
garvinhicking
Core Developer
Posts: 30022
Joined: Tue Sep 16, 2003 9:45 pm
Location: Cologne, Germany
Contact:

Re: pubsubhubbub

Post by garvinhicking »

Hi!

Query parameters to index.rss2 URLs might not work, especially if you have /feeds/index.rss&nocache=true, because it would need to be /feeds/index.rss?nocache=true - since URL rewriting is enabled here, it might happen that the parameters are not passed forth as GET to the actual script.

By using rss.php directly, you can make absolutely sure that all parameters come in, no matter if someone uses URL Rewriting or not.

Regards,
Garvin
# Garvin Hicking (s9y Developer)
# Did I help you? Consider making me happy: http://wishes.garv.in/
# or use my PayPal account "paypal {at} supergarv (dot) de"
# My "other" hobby: http://flickr.garv.in/
onli
Regular
Posts: 2830
Joined: Tue Sep 09, 2008 10:04 pm
Contact:

Re: pubsubhubbub

Post by onli »

Ah, ok. But I'm unsure: The hub compares the url with the subscribers. Isn't it possible that because the parameter the hub fails to see that this is a request for the same url without the parameter? More of the same problem with using directly the rss.php. And theoretically, the publisher (we) are even proposed to use last_modified-headers and etag. But the attached version contains links to the rss.php, but the more I think about it, the more I think that this can't work... the version above has a better chance to work :)

As the posts arrive at the moment, I guess that there need to be subscribers - in my testinstallation of a hub, this part didn't work. Would be great if you or someone else could test that.
Attachments
serendipity_event_pubsubhubbub.tar.gz
(2.47 KiB) Downloaded 432 times
garvinhicking
Core Developer
Posts: 30022
Joined: Tue Sep 16, 2003 9:45 pm
Location: Cologne, Germany
Contact:

Re: pubsubhubbub

Post by garvinhicking »

Hi!

Ah, I didn't know how that worked. But how does pubsubhubbub then handle different URLs? For many blog systems there are distinct URLs, and if they have parameters they would never really matchup as a single canonical URL?

But to test if it works at all, I'd start testing witht the rss.php URL instead of the 'rewritten' URLs?

Regards,
Garvin
# Garvin Hicking (s9y Developer)
# Did I help you? Consider making me happy: http://wishes.garv.in/
# or use my PayPal account "paypal {at} supergarv (dot) de"
# My "other" hobby: http://flickr.garv.in/
onli
Regular
Posts: 2830
Joined: Tue Sep 09, 2008 10:04 pm
Contact:

Re: pubsubhubbub

Post by onli »

I don't know how such a system could be handled by a hub. One would have to ignore all parameters and check the feeds anyway.

But yes, testing could be done with the rss.php. We only have to remember to use them to subscribe.
onli
Regular
Posts: 2830
Joined: Tue Sep 09, 2008 10:04 pm
Contact:

Re: pubsubhubbub

Post by onli »

Seems like i introduced a bug with the patch to the rss.php. 'channel_dat' also need to be unset from $entries. Otherwise and with current dev-build, a entry containing only "<" gets appended to the rss.

Code: Select all

diff -Nur serendipity-nightly/rss.php /var/www/rss.php 
--- serendipity-nightly/rss.php	2010-01-21 10:20:41.000000000 +0100
+++ /var/www/rss.php	2010-10-10 16:25:11.386515729 +0200
@@ -249,6 +249,7 @@
 $namespace_display_dat = $entries['display_dat'];
 $channel_display_dat = $entries['channel_dat'];
 unset($entries['display_dat']);
+unset($entries['channel_dat']);
 
 $serendipity['smarty']->assign_by_ref('metadata', $metadata);
 $serendipity['smarty']->assign_by_ref('entries', $entries);
garvinhicking
Core Developer
Posts: 30022
Joined: Tue Sep 16, 2003 9:45 pm
Location: Cologne, Germany
Contact:

Re: pubsubhubbub

Post by garvinhicking »

Hi!

Committed!

Regards,
Garvin
# Garvin Hicking (s9y Developer)
# Did I help you? Consider making me happy: http://wishes.garv.in/
# or use my PayPal account "paypal {at} supergarv (dot) de"
# My "other" hobby: http://flickr.garv.in/
blog.brockha.us
Regular
Posts: 695
Joined: Tue Jul 03, 2007 3:34 am
Location: Berlin, Germany
Contact:

Re: pubsubhubbub

Post by blog.brockha.us »

Hmm! This looks very interesting. Is there a public plugin already? Didn't find something in my plugin list, perhaps because my S9Y installation is outdated and needs the new hooks for that plugin?
- Grischa Brockhaus - http://blog.brockha.us
- Want to make me happy? http://wishes.brockha.us/
Post Reply