Corrupt file on 1.4 upgrade

rj · Post by rj » Thu Nov 13, 2008 4:53 am

My upgrade to 1.4 beta stalls because it says this

Verify Installation Integrity
templates/default/treeview/screen.css corrupt or modified: failed verification

All I did so far was a manual transfer the file again, it did not help.

What next?

Thanx!

RJ

rj · Post by rj » Thu Nov 13, 2008 5:02 am

oh yeah, and Im down...

I also pulled it out of the tar again and that didnt make a difference.

RJ

Post by **garvinhicking** » Thu Nov 13, 2008 1:50 pm

Hi!

Make sure your FTP client does a BINARY transfer of that file.

Gonna ask Judebert if he knows how this can happen

Regards,
Garvin

judebert · Post by **judebert** » Thu Nov 13, 2008 2:25 pm

I'm on it. Sorry this bit you, rj.

As background, I specifically wrote the verification code to prevent this sort of problem. We were getting so many users with problems from corrupted or incomplete uploads that we wanted some way to ensure that the uploads were correct. So I wrote some code and tested it on a few installations.

Obviously it still needs a more thorough vetting.

Thanks for trying 1.4, and I'll fix this problem as soon as I can figure it out.

rj · Post by rj » Thu Nov 13, 2008 2:39 pm

wow that did it!:) Thanx!

When I looked at the file before it was like one long line of code, not the usual look for a css file...

When I look at the file on my drive it is still that mess of course, but on the server it is all formated like a css file is suppose to look.

RJ

judebert · Post by **judebert** » Thu Nov 13, 2008 3:25 pm

That's what ASCII mode transfer is supposed to do.

The problem: Windows computers use \r\n to indicate the end of a line. Macs use only \r. Unix computers use only \n. When you transfer files between different computers (originally only possible with FTP or a floppy disk), they either showed up as one long line or with extra gunk at the end of each line.

The solution: FTP ASCII mode changes the end-of-line characters to the correct type for the receiving system.

The new problem: checksum programs really care, a lot, about that differing character.

The new solution: Judebert's checksum code converts all of them to spaces before taking the checksum.

The new new problem: Garvin's development system (that created the checksums to check) and your server (that actually checked the file against the checksum) got two different answers.

The new new solution: Judebert fixes the checksum code, as soon as he figures out what's going on. (Oh, it's never good when you start referring to yourself in the third person.) Until then, you can transfer that file in binary mode.

Could you tell me what version of PHP you're running? It might help.

Post by **garvinhicking** » Thu Nov 13, 2008 3:37 pm

Hi!

On a sidenote: I've fixed the screen.css file that seems reponsible, to not look garbaged. It used an odd combination of unix and windows linebreaks which somehow seemed to have confused the checksum generation code. That's all I know

Go judebert, go judebert! *checking out my cheering-pom poms*

Regards,
Garvin

rj · Post by rj » Thu Nov 13, 2008 4:07 pm

php version 4.3.11

judebert · Post by **judebert** » Thu Nov 13, 2008 7:27 pm

Thanks, rj. And an especially big thanks to Don Chambers for helping me out with this one. Don took a tester I made and ran it on a server using PHP 4.3.11.

Turns out it's not PHP at all. The problem is that some FTP clients are incorrectly mangling the file during the ASCII transfer. This file has Mac-style linefeeds; the FTP server is supposed to convert them to its local linefeed style. Instead, it's removing the linefeeds altogether. That's not cool!

You can try out the tester, too; just download http://judebert.com/progress/media/s9y/sumcheck.zip to your local machine, unzip it, and put the files in a single directory on your web server.

It's important that you upload the four files correctly. The sumcheck.php and screen.ascii.css should be uploaded in ASCII mode. The screen.bin should be uploaded in BINARY mode. The screen.css should be uploaded in whatever mode your FTP client thinks best. The files are named so that your FTP client should pick the correct mode for each.

Then just visit sumcheck.php, and you'll get a nice XHTML compliant webpage explaining everything. Here's an example.

The Solution (impatient people skip to here)
Hoo boy, was that file ever hosed; I'm surprised it ever worked. Garvin already did the right thing by fixing the file. You can download the next nightly build and be happy.

Sorry for the inconvenience, and thanks for helping us find and fix this problem!

rj · Post by rj » Thu Nov 13, 2008 7:39 pm

What I dont understand is that I thought FTP transfered in binary by default... Or is it that FTP knows to transfer binararily

for some extentions like exe and then do it ASCII for others, like css?

judebert · Post by **judebert** » Thu Nov 13, 2008 8:01 pm

It depends on what you use. Command-line FTP clients usually transfer by binary by default, but you've got to manually type in every file you want to transfer. That's such a big pain that most people use graphical clients.

The graphical clients usually check the file extension to determine whether to use ascii or binary transfer types. Since CSS is an ascii type, that's probably how it got uploaded. And then the server hosed it by removing all the linefeeds.

That's why the files are named the way they are in my tester. The .bin extension is almost always recognized as binary; the .php and .css extensions are almost always recognized as text. That way you can just upload with a graphical FTP client and everything should go as planned (although you'll want to check the client's log to make sure).

Post by **garvinhicking** » Fri Nov 14, 2008 11:15 am

Hi!

So *actually* the checksum code workde properly and indeed warned about a damaged file, because INDEED the file was badly uploaded!

Hooray!

Regards,
Garvin