3D card

[Grifter]

The text files are on a very over-used server at Stanford (http://folding.Stanford.edu/), and they are each 500KB. It takes a while to grab them if it's also loading the PHP too... The servers are Linux MP servers, if I am not mistaken.

First... MP doesn't make a difference all the time.. especially if it's an intel PPro200 dual or some such crap. ?There are plenty of slow MP boxes out there... no matter what OS they run... ?

How do you figure that pulling in two files that are almost 1MB off of a remote server is faster then pulling two lines of data from a MySQL server? I have never heard that one before!

Ok, that's a simple one....

I'm going to assume that you are parcing out the 1 meg worth of data into the mysql database when you get it... because you mentioned that you are only pulling two lines... ?you could just as easily parse out that 1 meg of data into flat files and serve it the same way without having to use mysql.... ?

Somewhere you are slicing off those 1 meg files into smaller bites.... you can do it the same using flat files... ?and 1 meg is not a lot of data to parse through. ?You are NOT parcing that same data on the fly with mysql and doing it real time. ?You are taking it and storing it in the database for future use.... well, if you take it and store it in flat files for future use... your scripting language should be able to get them faster as flat files then using sql....

[SiGmA_X]

Nope. You are wrong. The data I pull is near the middle. It takes it 20 seconds to grab the 3 lines of data from the file and then to print it. It takes my MySQL PHP page a whole 15 seconds to file the data away in to the MySQL, and then it takes 2/10 of a second to grab the data.

I'll take my way and use the MySQL.

[.vooDoo.]

well this thread went way off subject,
and btw, does anyone know what these 2 nerds are talking about, lol

[Grifter]

Nope. You are wrong. The data I pull is near the middle. It takes it 20 seconds to grab the 3 lines of data from the file and then to print it. It takes my MySQL PHP page a whole 15 seconds to file the data away in to the MySQL, and then it takes 2/10 of a second to grab the data.

I'll take my way and use the MySQL.

Let's see Rom.... it takes your system 20 seconds to grab the data and then 15 seconds to load it into mysql... ?that's 35 seconds. ?So if PHP times out at 30, you are doin this offline, not on the fly. ?Right?

Ok, so then, you have a choice... the data is all there on your server... it's either in flat files or a database.... you follow? ?Now, when someone hits the web site, php submits a request.... it can either just go to the flat file and get it and then render the page, or it can compose a sql request... send that to another program (mysql in this case), the database can then look up the data and send it back to php, which then can render the page... ?which one sounds more simple to you (I didn't even bring up formats...)? ?Which do you think would be faster for php to do? ?Which do you think will be more stable (two programs that have to work together, or just one of the two)??

There are reasons for using databases behind the scenes... you just haven't made a case for it. ?Complex searches and joins... lookups on multiple tables... running calculations (not just simple ones)... etc. ?Those are reasons to use databases. ?Well... that and very very large volumes of data, but at that point, you are having to normalize the databases and it's as much for backup purposes. ?Remember... the more data you put in the database, the slower it will get. ?While if architected right, the flat files will keep the same speed (because you are relying on file naming and directory structure). ?

So if I'm wrong, you sure better explain to me how adding a second app speeds something up when you are talking about simple text data??

[SiGmA_X]

Let's see Rom.... it takes your system 20 seconds to grab the data and then 15 seconds to load it into mysql...  that's 35 seconds.  So if PHP times out at 30, you are doin this offline, not on the fly.  Right?

Ok, so then, you have a choice... the data is all there on your server... it's either in flat files or a database.... you follow?  Now, when someone hits the web site, php submits a request.... it can either just go to the flat file and get it and then render the page, or it can compose a sql request... send that to another program (mysql in this case), the database can then look up the data and send it back to php, which then can render the page...  which one sounds more simple to you (I didn't even bring up formats...)?  Which do you think would be faster for php to do?  Which do you think will be more stable (two programs that have to work together, or just one of the two)??

There are reasons for using databases behind the scenes... you just haven't made a case for it.  Complex searches and joins... lookups on multiple tables... running calculations (not just simple ones)... etc.  Those are reasons to use databases.  Well... that and very very large volumes of data, but at that point, you are having to normalize the databases and it's as much for backup purposes.  Remember... the more data you put in the database, the slower it will get.  While if architected right, the flat files will keep the same speed (because you are relying on file naming and directory structure).  

So if I'm wrong, you sure better explain to me how adding a second app speeds something up when you are talking about simple text data??

This simple text document is HUGE. It takes about 20 sec for the PHP script to find the required data, then it outputs it. This means that it will take 20 seconds every time a page loads! With the MySQL, it takes about 20 seconds to file the data away in to the tables. Then, when it pulls the data out, it only takes about 2/10's of a second.

Another reason that you left out is for easier and faster changes of data files. You just have to modify one Row instead of a entire text file. I personally like the MySQL route better, as do many PHP programmers. I have noticed that many PERL programmers prefer text documents for the same stuff...

[Grifter]

Rom, try real hard to understand this... you are still comparing apples to oranges....

With the mysql process, you are parcing out the data that you need from the old text files OFFLINE.  Then you compare it to PHP going through the old text files ON THE FLY.  What I am talking about is you pull the important three lines into a NEW TEXT FILE, just like you do when you put it in the database... just keeping it as a flat file.  Then, when PHP has to do it's thing... it just has to pull up the correct text file (which should only be about 1k in size).... not search a 500k file, because that is history.  Changing these new text files is easier then changing a record in mysql... because you don't need to know SQL to do it... just the file names.

As for why people that use PHP like mysql so much is because PHP was developed with mysql in mind.  It has many shortcuts to it.  It's faster then PERL because of these shortcuts (PHP was written just for the web.. PERL is a bigger and much more powerful scripting language).  PHP with mysql is just easier for some people to setup, since it's already done for them by projects like PHP-NUKE....  it doesn't make it better.  It's less reliable could be faster if they didn't always rely on a database for the small stuff.

[Brain]

well this thread went way off subject,
and btw, does anyone know what these 2 nerds are talking about, lol

not a single fucking clue
<--me
all i know it they were talking about websites at one point