spartacus fails to mirror htaccess

[Timbalu]

I am just sitting to update the (old) FCKEditor plugin and found some issues to solve.

The one not being able to, is Spartacus not downloading, copying or writing the essential .htaccess file, which is in GitHub repo definitely and also in the manual downloaded plugin via Spartacus board.

Mirror is default: 'Netmirror.org', but even 's9y.org' does the same. This was with Spartacus version: 2.23 on 1.6.1 and I have the same with Spartacus 2.29 and github as mirror with 1.7.

Any ideas?

I think this is quite essential for some plugins!

Edit:
Also the manual downloaded plugin has a CVS/ dir, which is not in the spartacus online downloaded plugin. But there isn't any scripting removement of 'CVS' or '.htaccess' in serendipity_event_spartacus.php file...

[garvinhicking]

Hi!

Probably files beginning with a "." are not fetched when downloading via HTTP, yes. I don't think this can be rectified easily. Maybe the plugin could write the .htaccess on its own (however it might not have write privileges after a spartacus download, that could be an issue).

Another way would be to create a second file called "htaccess" without the dot. When spartacus downloads a file named this, it would instead save it as ".htaccess". A bit hacky, though.

Regards,
Garvin

[Timbalu]

Hmm, the second idea could be a way to go, even if a bit hackish...

But I would prefer to fix spartacus for this ... I remember glob() using exceptions eg. glob($dir."{*,.htaccess}", ...).

But why was this not discovered until now? I counted 11 additional plugins making use of a .htaccess file, probably to set rewriteEngine off. There are prominent Plugins in this list, like staticpage.

[yellowled]

Another way would be to create a second file called "htaccess" without the dot. When spartacus downloads a file named this, it would instead save it as ".htaccess". A bit hacky, though.

Pretty common practise, however. I've seen this in other CMS, e.g. calling it ht.access etc.

YL

[Timbalu]

I am struggling with this issue... does this (via http) mean, spartacus or whatever does not fetch the zip of that additional_plugin and instead fetches files and subdirs one by one from any of these mirror places?
I have no clue, how the sync with github actually stores (does it store zips, single files and dirs or just the xml information?) update ready plugins on or to these mirrors. Is the .htaccess (eg. staticpages) delivered to them or have we got an issue with the .gitignore file on github blacklisting .htaccess* transfers? Garvin...?

Well, ...often writing it "to paper" is clearing (some) things up...
http://netmirror.org/serendipity/additi ... taticpage/
shows it "~must" be the gitignore...

[garvinhicking]

Spartacus never fetched a zip, it always called each URL via HTTP and saved the contents.

Whatever is inaccessible via HTTP cannot be saved. Maybe netmirror previously served up also .htaccess files. Everyone who downloaded the ZIP would have the file, and also everyone who used CVS or Github.

Regards,
Garvin

[Timbalu]

Yes, but these are the minors (IMO). Now I am pretty sure it is the .gitignore file, since no mirror (including github via spartacus) fetches the .htaccess file. So it can truly never arrive the user (at least via spartacus) when updating plugins. The zips are not involved. I can't see any direct reason why this rule should be kept in there, anyhow.

I would pled to delete the .htaccess* gitignore rule and push a bump update to all plugins including a .htaccess. Then we could see tomorrow morning if the sync properly worked.

btw. https://github.com/s9y/Serendipity/blob ... s.php#L414
should IMHO be

Code: Select all

$curl_result = curl_exec($curl_handle);

EDIT: Huh, strange. I looked falsely into s9y/serendipity/.gitignore, but additional_plugins has its own without any htaccess rule... - but it might be in one of the other files including some ". -not" (or equal) command, or is in emerge_spartacus.php (does this need to sync with the updates event_spartacus has gotten over the time. like added github for download mirror?)...

[garvinhicking]

Hi!

Yes, the curl_result thing looks like a bug as well, would you like to commit?

.htaccess is only blocked for the main Serendpity repository, where it needs to stay..htaccess is not fetched via HTTP by most servers, so this is a problem of the filename we need to change like in my first post.

However you are also right that emerge_spartaucs currently filters anything with a "." in front of it. I've just changed this, I think this will put the files in the xml output. But I really doubt that the files then can be downloaded. In fact, it might even lead to trouble because then files that the plugin may not really completely need will not get downloaded and then spartacus could report an error. In that case we would need to revert this patch...

So I think renaming .htaccess to htaccess and changing spartacus to then re-rename such files is the only really fully compatible way to go.

Netmirror does not allow to download it:
http://netmirror.org/serendipity/additi ... /.htaccess

Github does though:
https://raw.github.com/s9y/additional_p ... /.htaccess

SF.net does as well:
http://php-blog.cvs.sf.net/viewvc/php-b ... sion=1.999

s9y.org does not:
http://s9y.org/mirror/additional_plugin ... /.htaccess

Regards,
Garvin

[garvinhicking]

Hi!

I removed it again; having those . files in there will create more trouble due to not downloadable files and HTTP 40x errors with s9y or netmirror as an mirror. The only solution I see is to go the file-rename route.

Regards,
Garvin

[Timbalu]

How bad!!
Thanks for looking up, Garvin. (*)

Then, how shall we do this file-rename route, that it will be done by the automatic spartacus install, without causing writable problems ...maybe via function example() or better install()?

Edit; Would this be a way to go for all these plugins?

Code: Select all

    function install() {
        $dotfile  = '.htaccess';
        $rewfile  = 'htaccess';
        $thispath = dirname(__FILE__);
        if (file_exist($thispath . '/' . $rewfile)) {
            @unlink($thispath . '/' . $dotfile);
            @rename($thispath . '/' . $rewfile, $thispath . '/' . $dotfile);
            @umask(0000);
            @chmod($thispath . '/' .  $dotfile, 0664);
            @unlink($thispath . '/' . $rewfile);
        }
    }

(*) Well, we could get rid of netmirror and s9y as mirrors. Hmm, or work around those two for the dotfile removement in the emerge script and then have a switch for this in the install route... Stop!

[Timbalu]

Alright one night with a hat of sleep later..., these 4 pasted links show, that the .htaccess file is synct to the mirrors though, which my first (netmiror.org) link didn't show, right?!
So we have netmirror and s9y.org which should also have it, but deny to show and access it via HTTP, right?! This should be easy(?) to solve, ... maybe within a Servers .htaccess file in or above /additional_plugins like

Code: Select all

  <Files .htaccess>
    allow from all
  </Files>

or

Code: Select all

  <Files ".htaccess">
    Order Allow,Deny
    Allow from all
  </Files>

Can you check this? If that runs successfully the 'xml allow dotfile' commit in emerge* should work...

Edit: Btw, is there a checksum check for spartacus downloads to secure this a little bit more?

[garvinhicking]

Hi!

Quite a good idea with the <Files> directive. I just committed it to see if it works on the 2 mirrors on the next run tomorrow.

Regards,
Garvin

[Timbalu]

Welcome back, Garvin!

Apart from non-bumped versions for the sync to trigger, this won't do, I assume. What I meant, was to put this htaccess file on netmirror.org/*/additional_plugins/.htaccess (same for S9y.org), or one dir above, to allow the full HTTP access for Spartacus. We do not want to have this "Allow from all" in Serendipity installations, though, do we?!

[garvinhicking]

Hi!

Of course. Those files are synced by emerge_spartacus. When the .htaccess files are callable on the servers, I will then patch emerge_spartacus to also include .files in the created XML files.

Being able to download a .htaccess from an actual Serendipity installation wouldn't do harm, the file is not meant to be user-editable (and if it were user-edited, they would themself remove that line...)

So I think all is well. (If netmirror & s9y.org actually interpret this file. s9y.org is nginx, it could be that this does not work at all).

[Timbalu]

Ahhh, fine!
So the mirror sync is independently from xml sync, right?! This wasn't clear for me.
Alright, being Admin-User editable is no harm too. I was just in worry we could open a door with that.

But since s9y.org is under your control, I presume, this could easily be patched inside the servers config, if not.

Edit:
http://netmirror.org/serendipity/additi ... icpage.php is fetchable and readable, but not accessible, which is meant to be like this, since it is a mirror. So I am pretty sure they will not allow a mirrors/*/.htaccess file being interpreted by the server too, which would be needed, to allow reading this file via HTTP. That is why I suggested to change that further up, before we are in mirror mode (http://netmirror.org/serendipity/.htaccess).