Search Help Board

PHP FAQ
PHP Articles
PHP Help
Bulletin Board

PHP Manual (NEW!)
First Time PHP'ers
Help with programming
Sql assignment help
PHP Homework Help


 

Previous Page Page 3 of 3  

Frequently Asked Questions

  1. Create a Flat File Database
    Flat file databases are when you store your data in a normal text file.  They tend to be easy to start but hard to maintain as they grow more complex.  Other databases (e.g. relational databases) are often easier to search through and can handle multiple people accessing them at one time; some are available for free, such as MySQL and PostgreSQL.  If you cannot use one of these databases, here is the code to create your own using files.
     
    One incredibly important part of a flat file database is file locking.  If you don't lock a file that several people have access to, you can get situations where you loose data - it just dissapears.  For example, two users try and update data in the database at same time. Both users read in the old, unchanged lines and one user changes one line while the other user changes a different line.  When the first user writes to the database only his change is saved.  When the second user writes his data it only contains his change, so the change of the first user dissappears and is lost. 
     
    Because of this, we include what is called file locking.  When you lock a file, you tell the operating system that no one else can alter it.  Then your changes will not be erased because no one else can write to it.  Other users have to wait until the file is no longer locked before using it. 
     
    One of the advantages of using a database is their superior locking.  At a minimum, databases have file locking built in, however, most databases have what's called "record locking".  The problem with file locking is that no one else has access to that file (e.g. the actor file/table) until the one user is done with it.  With record locking, a user "locks" one record (or line) and other users can alter other records (or lines) in the database.  Therefore, there are less users sitting around waiting to access the database.  Again, this is very optimized in any database.
     
    By using php's flock(), we don't have to worry about many details involved in ensuring locks on files.  For example, if the user starts updating the database and clicks on the 'Stop' or 'Back' button before we can unlock it, thus leaving the file permanently locked. 

     
    When you go to create your database, the first thing that you have to do is sit down with a pen and paper and think long and hard about what you want stored in your database, how you want to access it, and how you might have to access it in the future.  This is the most important part of any database creation.  What you decide here will result in how the data is stored in the file.
    e.g.
    Perhaps your "Cool Actors" database will look like this:

    actorId|actorName|country|...

    So the data will go in like this:

    1|Graham Green|Canada|...
    2|Robbie Coltrane|England|...
    3|Nicholas Campbell|Canada|...

    We will be copying the new changes to a temporary file and then we will copy the temporary file over top of the old database file.  But, in order to copy we have to unlock the file.  If we unlock the file, another user can read the old one before we get the copy finished, causing the same locking error as described above.  Because of this we are going to lock a separate file that we can keep locked while the copying is occuring.  However, we are going to use php's flock() on that file so that we don't have to write all the annoying code to ensure that it never gets locked permanently.
     

    PHP Code:

    <?php

    Class FlatFileDatabase
    {
      var $dbFile       = "";
      var $dbTmpFile = "tmpDb.txt";
      var $lockFile  = "actor.LCK";      // our lock file.
     

      Function FlatFileDatabase($dbFile)
      {
        $this->dbFile = $dbFile;
      } /* FlatFileDatabase */
     

      Function find_record($id, $fd, $fd2="")
      {
        /*
         * Find the record/line in the file.
         *   Don't use fopen() here because you will want to maintain the
         *   file lock and position (offset) in the file.
         *
         * PARAMETERS:
         *   id    the id to find.
         *   fd    the file pointer/descriptor to use.
         *   fd2  the file pointer/descriptor to the tmp file.
         *
         * RETURNS:
         *   <string>  the line found on success,
         *   false      not found,
         *   -1           on error.
       */

        while ( ($line = fgets($fd, 10000))) {
          $tmp = explode("|", $line);

          // see if it's the one we want, if we're looking for one.
          if ($id && ($tmp[0] == $id))
            return $line;

          // if not, and we're updating the file, write it to the tmp file.
          if ($fd2) {
            if (fwrite($fd2, $line) == -1) {
              echo "ERROR (find_record): unable to write line to tmpDb for '$tmp[0]'.<br />\n";
              return -1;
            }
          }
        }//while

        return false;
      } /* find_record */
     

      Function copy_new_db()
      {
        /*
         * Replace the original db file (now useless) with the newly
         *   updated version.
         *
         * NOTE: if you're having errors, check the file & directory permissions.
         *
         * PARAMETERS:
         *   <n/a>
         *
         * RETURNS:
         *   true     on success,
         *   false   on error.
         */
        $ret = true;

        // remove the original db file.
        // copy will actually rewrite the file, but I put this here so
        // that you can understand what's going on better.
        $ret = unlink($this->dbFile);

        // make the new tmp file the original file by copying it.
        $ret = copy($this->dbTmpFile, $this->dbFile);
        if (!$ret)
          echo "ERROR (copy_new_db): unable to copy '$this->dbTmpFile' to '$this->dbFile'.<br />\n";

        // clean up our garbage by deleting the tmp file.
        if ($ret) {            // don't delete it if the copy didn't work.
          $ret = unlink($this->dbTmpFile);
          if (!$ret) {
            echo "WARNING (copy_new_db): unable to delete '$this->dbTmpFile'.<br />\n";
          }
        }

        return $ret;
      } /* copy_new_db */
     

      Function lock_file($fd, $fcnName, $id)
      {
        /*
         * Lock the file.
         *
         * PARAMETERS:
         *    fd           file descriptor to lock
         *    fcnName name of the function calling this one.
         *    id            id we are looking for.
         *
         * RETURNS:
         *    true      on success,
         *    false    on error.
         */
        
        $i = 0;
        while (!flock($fd, LOCK_EX)) {    //20 seconds
          if ($i > 2) {    //3 tries = 60 sec.
            fclose($fd);
            echo "ERROR ($fcnName): unable to obtain file lock for '$id'.<br />\n";
            return false;
          }
          $i++;
        }
        return true;
      } /* lock_file */

     

      Function close_file($fd)
      {
        /*
         * Unlock and close the file.
         *
         * PARAMETERS:
         *    fd           file descriptor to lock
         *
         * RETURNS:
         *    true      on success,
         *    false    on error.
         */
        
        flock($fd, LOCK_UN);
        fclose($fd);
        
        return true;
      } /* close_file */

     

      Function AddNewRecord($data)
      {
        /*
         * Add a new record/line to the bottom of the file.
         *
         * PARAMETERS:
         *   data   an array of the data to be added.
         *               array(1, "graham green", "canada", ...)
         *
         * RETURNS:
         *   true     on success,
         *   false   on error.
         *   -1       unable to obtain lock.
         */
        $ret = true;

        if (!$data) {
          echo "ERROR (AddNewRecord): parameter missing.<br />\n";
          return false;
        }

        // assemble the record.
        $string = implode("|", $data) . "\n";

        // lock the files.
        $fdLck = @fopen($this->lockFile, "w");
        if ($fdLck === false) {    // exactly false
          echo "ERROR (AddNewRecord): unable to open '$this->lockFile'.<br />\n";
          return false;
        }
        if (!$this->lock_file($fdLck, "AddNewRecord", $id))
          return -1;
        $fd = @fopen($this->dbFile, "a");
        if ($fd === false) {    // exactly false
          $this->close_file($fdLck);
          echo "ERROR (AddNewRecord): unable to open '$this->dbFile'.<br />\n";
          return false;
        }
        // locking this one is probably redundant, but better
        //  redundant and safe than sorry.
        if (!$this->lock_file($fd, "AddNewRecord", $id)) {
          $this->close_file($fdLck);
          return -1;
        }
        
        // save it.
        if (fwrite($fd, $string) == -1) {
          echo "ERROR (AddNewRecord): unable to write new string to db '$string'.<br />\n";
          $ret = false;
        }
        
        $this->close_file($fd);
        $this->close_file($fdLck);

        return $ret;
      } /* AddNewRecord */
     

      Function ReadARecord($id)
      {
        /*
         * Read a record/line from the database.
         *   Sometimes 'read' file locking can be added to allow multiple people to read
         *   a file but to prevent anyone from writing to it and altering it.
         *   We are not going to use that here because it will not be a problem.
         *
         * PARAMETERS:
         *   id   the id to of the record/line to get.
         *
         * RETURNS:
         *   <array> of the data from the record/line on succes,
         *   false     no line found,
         *   -1         on error.
         */

        if (!$id) {
          echo "ERROR (ReadARecord): parameter missing.<br />\n";
          return -1;
        }

        $fd = @fopen($this->dbFile, "r");
        if (!$fd) {
          echo "ERROR (ReadARecord): unable to open the files for '$id'.<br />\n";
          return -1;
        }

        $string = $this->find_record($id, $fd);
        fclose($fd);

        if ($string && ($string != -1))
          $data = explode("|", $string);
        else
          $data = $string;

        return $data;
      } /* ReadARecord */
     

      Function DeleteARecord($id)
      {
        /*
         * Delete a record from anywhere in the file - beginning, end, middle.
         *
         * Due to the nature of current file systems, it is impossible to
         *   just "delete" data out the middle of a file because the file
         *   length will change. We have to copy the first part to a temporary
         *   file without copying the line we want to delete, then copy the rest
         *   of the file over.
         *
         * PARAMETERS:
         *   id   id of the record/line we want to delete.
         *
         * RETURNS:
         *   true    on success,
         *   false  record not found,
         *   -1      on error.
         */
        $ret = true;

        if (!$id) {
          echo "ERROR (DeleteARecord): parameter missing.<br />\n";
          return -1;
        }

        // lock the files.
        $fdLck = @fopen($this->lockFile, "w");
        if ($fdLck === false) {    // exactly false
          echo "ERROR (DeleteARecord): unable to open '$this->lockFile'.<br />\n";
          return -1;
        }
        if (!$this->lock_file($fdLck, "DeleteARecord", $id))
          return -1;
        $fd = @fopen($this->dbFile, "r");
        if ($fd === false) {    // exactly false
          $this->close_file($fdLck);
          echo "ERROR (DeleteARecord): unable to open '$this->dbFile'.<br />\n";
          return -1;
        }
        if (!$this->lock_file($fd, "DeleteARecord", $id)) {
          $this->close_file($fdLck);
          return -1;
        }
        $fd2 = @fopen($this->dbTmpFile, "w");
        if ($fd2 === false) {    // exactly false
          $this->close_file($fdLck);
          $this->close_file($fd);
          echo "ERROR (DeleteARecord): unable to open '$this->dbTmpFile'.<br />\n";
          return -1;
        }
        if (!$this->lock_file($fd2, "DeleteARecord", $id)) {
          $this->close_file($fdLck);
          $this->close_file($fd);
          return -1;
        }

        // find the record we want to delete.
        $string = $this->find_record($id, $fd, $fd2);
        if (!$string) {
          echo "WARNING (DeleteARecord): unable to find record for '$id'.<br />\n";
          $ret = false;
        }
        else if ($string == -1) {
          echo "ERROR (DeleteARecord): error occured finding record for '$occurred'. <br />\n";
          $ret = -1;
        }

        // we delete it by not writing it to the tmp file:
        /* DO NOTHING */

        // then we copy the remaining records/lines from the original db file to the
        // tmp file.
        if ($ret > 0) {
          $ret = $this->find_record("", $fd, $fd2);
          if ($ret == -1)
          echo "ERROR (DeleteARecord): unable to write rest of database for '$id'.<br />\n";
        }

        // close the files we want to copy.
        $this->close_file($fd);
        $this->close_file($fd2);

        // replace with new database file.
        if ($ret > 0) {
          $ret = $this->copy_new_db();
          if (!$ret) {
            echo "ERROR (DeleteARecord): unable to replace new database with old database for '$id'.<br />\n";
            $ret = -1;
          }
        }

        // close the lock file.
        $this->close_file($fdLck);

        return $ret;
      } /* DeleteARecord */
     

      Function ChangeARecord($id, $newData)
      {
        /*
         * Change a record anywhere in the file - beginning, end, middle.
         *
         * Due to the nature of current file systems, it is impossible to
         *   just "change" data in the middle of a file because we
         *   cannot gaurantee that the data we want to replace is the
         *   exact same length as the original data. We have to copy
         *   the first part to a temporary file without copying the line we
         *   want to change, then copy the new line, then copy the rest of the file
         *   over.
         *
         * To be completely modular, this code should be incorporated into
         *   the above function, but keeping it separate makes it easier to
         *   understand.
         *
         * PARAMETERS:
         *   id             the id of the record/line to replace.
         *   newData   array of new data to put into the database:
         *                       array(1, "graham green", "canada", ...)
         *
         * RETURNS:
         *   true    on success,
         *   false  record/line not found,
         *   -1      on error.
         */
        $ret = false;

        if (!$id || !$newData) {
          echo "ERROR (ChangeARecord): parameter missing (id $id, newData $newData).<br />\n";
          return -1;
        }

        // lock the files.
        $fdLck = @fopen($this->lockFile, "w");
        if ($fdLck === false) {    // exactly false
          echo "ERROR (ChangeARecord): unable to open '$this->lockFile'.<br />\n";
          return false;
        }
        if (!$this->lock_file($fdLck, "ChangeARecord", $id))
          return -1;

        $fd = @fopen($this->dbFile, "r");
        if ($fd === false) {    // exactly false
          $this->close_file($fdLck);
          echo "ERROR (ChangeARecord): unable to open '$this->dbFile'.<br />\n";
          return false;
        }
        if (!$this->lock_file($fd, "ChangeARecord", $id)) {
          $this->close_file($fdLck);
          return -1;
        }
        $fd2 = @fopen($this->dbTmpFile, "w");
        if ($fd2 === false) {    // exactly false
          $this->close_file($fdLck);
          $this->close_file($fd);
          echo "ERROR (ChangeARecord): unable to open '$this->dbTmpFile'.<br />\n";
          return false;
        }
        if (!$this->lock_file($fd2, "ChangeARecord", $id)) {
          $this->close_file($fdLck);
          $this->close_file($fd);
          return -1;
        }

        // find the record we want to delete.
        $ret = $this->find_record($id, $fd, $fd2);
        if (!$ret || ($ret == -1))
          echo "ERROR (ChangeARecord): unable to find record for '$id' ($ret).<br />\n";
        else
          $ret = true;

        // copy the new data in to replace the old data.
        if ($ret > 0) {
          $newRecord = implode("|", $newData) . "\n";
          if (fwrite($fd2, $newRecord) == -1) {
            echo "ERROR (ChangeARecord): unable to write new record for '$id'.<br />\n";
            $ret = -1;
          }
        }

        // then we copy the remaining data from the original db file to the
        // tmp file.
        if ($ret > 0) {
          $ret = $this->find_record("", $fd, $fd2);
          if ($ret == -1)
            echo "ERROR (ChangeARecord): unable to write rest of database for '$id' (ret $ret).<br />\n";
          else
            $ret = true;
        }

        // close the files.
        $this->close_file($fd);
        $this->close_file($fd2);

        if ($ret > 0) {
          $ret = $this->copy_new_db();
          if (!$ret) {
            echo "ERROR (ChangeARecord): unable to replace new database with old database for '$id'.<br />\n";
            $ret = -1;
          }
        }

        // close the lock file.
        $this->close_file($fdLck);

        return $ret;
      } /* ChangeARecord */
     

    }//class

    ?>

     
  2. Edit/Delete Data in the Middle Of A File
    Editing data in the middle of a file is not as straight forward as one would think.  The way the current file systems work, you cannot replace the data unless it is the exact same length.  Because of this, you have to read all the data in, edit the data, and write it all out again.  There are two ways of doing this, one for small files (i.e. most files) and one for large ones (i.e. very large files).

    Small Files:
    Small in this case is a relative term.  Essentially you can read the entire file into one variable or array, edit it, and write it out again.  However, if the file is too large the processing could slow down and the browser connection may time out.  If the file is even larger, PHP may not be able to handle it properly.  This will not usually be an issue, though.  One example is:

    PHP Code:

    <?php

    $fileArray = file($filename);
    if ($fileArray === false) {    // exactly false
      echo "ERROR : unable to open '$filename'.<br />\n";
      exit;
    }
    for($i = 0; $i < count($fileArray); $i++) {
      $tmp = explode("|", $fileArray[$i]);

      // see if it is the one we want, if we are looking for one.
      if ($id && ($tmp[0] == $id)) {
        // change it.
        $fileArray[$i] = implode("|", $newLine);
        
        // or delete it.   (this will ruin the for loop count)
        unset($fileArray[$i]);
        break;
      }
    }

    //write the file.
    $fd = @fopen($filename, "w");   // write over it.
    if ($fd === false) {    // exactly false
      echo "ERROR : unable to open '$filename'.<br />\n";
      exit;
    }
    foreach($fileArray as $item)
      fwrite($fd, $item);
    fclose($fd);

    ?>


    Large Files:
    However, if you think that the file size will be a concern, try reading in each line of data and copying it into a tmp file.  When you read in the line you want to change or delete write the new data to the temporary file or do not write the line to delete it.  Then copy the rest of the data in the file to the tmp file.  After that you delete the original file and rename the tmp file to the same name as the original file.
    If multiple users will be accessing the file, you will also need to include file locking.
     
    For example code, refer to Function ChangeARecord and Function DeleteARecord in the above code.

     

Previous Page Page 3 of 3