🚀 go-pugleaf

RetroBBS NetNews Server

Inspired by RockSolid Light RIP Retro Guy

Article View: borland.public.jbuilder.java.api
Article #932

Re: OutOfMemoryError

#932
From: Alan Shiers
Date: Sun, 07 Jul 2002 01:27
641 lines
22251 bytes
Hi Owen,

I've been slaving away trying to figure things out for these functions
and I believe I've solved the case.  At least to my satisfaction for
now.

My main dilemma was this while loop:

while((n = fin.read(buf)) >= 0)
{
    rq.output.write(buf);
}

I found this certainly works on the client side when reading in a file
and sending the data out the wire, but it was a "no go" on the receiving
end on the server app.  When implemented there the while loop just
wouldn't exit the loop.

Therefore, I had no choice but to reconsider using the for loop in an
inventive manner.  I am always sending the size of each file anyway so I
thought to use it to help terminate the for loops.

Since I'm sending arrays of data that are 128bytes in size for files 1Mb
or smaller, and 256bytes for those files larger than 1Mb, I decided to
encorporate some simple calculations in the receiveFile() method on the
server app. 

I'm simply deviding the file size by the array size whose result I'm
using as part of the conditional statement of a for loop. This gets the
majority of the data.  Then I use a second for loop to get the remaining
data to  ensure I'm getting and retaining the full file size and data. 
Because I am sending arrays of data, the last array is going to have
some
blank elements, so I collect those as well but don't do anything with
them.  This is necessary, and I won't go into the reasons here.

The main thing is that it works!

For your assessment, following are the same methods I had sent earlier
revised. If you see anything in there that could use some adjustment or
improvement, let me know.

Alan

***************************************************

On the server side app:

private void receiveFile(String pathFilename)
  {
    //System.out.println("Incoming path: " + pathFilename);
    //this method is called when a file is being uploaded to the server
    //and is to be saved at the specified directory on the server.

    /*Before we save any file, we need to ensure there is
     *enough free space available in the user's
     *home directory to accomodate it depending on
     *how much initial space was allotted to the user via
     *his Member Profile. If there is no space left,
     *we simply return out of here.
     */

    int fileLength = 0;
    try
    {
      //System.out.println("Trying to read int");
      fileLength = input.readInt();
    }
    catch(IOException ioe){ioe.printStackTrace();}

    boolean enoughSpace = isEnoughSpaceAvailable(fileLength);
    //System.out.println("enoughSpace = " + enoughSpace);
    if (enoughSpace == false) return;

    File myFile = new File(pathFilename);
    byte[] buf;
    byte[] buf2;
    FileOutputStream fileOut = null;

    try
    {
       myFile.createNewFile(); //create the file
       if(myFile.exists()) //Method exists() throws a SecurityException
if
                          //there is an access violation.
       {
          try
          {
            fileOut = new FileOutputStream(myFile);
          }
          catch(FileNotFoundException fnfe)
          {
            System.out.println("receiveFile() function -
FileNotFoundException\n");
            fnfe.printStackTrace();
          }

          /*1 megabyte (M / MB) = 2^20 bytes = 1,048,576 bytes
           *We'll receive the data in chunks of 128 bytes if the
           *file is less than 1Megabyte. Anything larger than
           *1Megabyte, we'll receive in chunks of 256bytes.
           */

          if(fileLength < 1048576 && fileLength > 128)//the file is less
than 1Mbyte
          {
            buf = new byte[128];
            //Calculate the number of full arrays
            //are needed to receive the majority of the data.
            int x = fileLength / 128;
            int sum = x * 128;
            //System.out.println("sum = " + sum);
            //load up the array

            for(int i = 128; i <= sum; i = i + 128)
            {
              input.read(buf);
              fileOut.write(buf);
            }
            //Now, calculate the remaining number of bytes
            //required to receive the data to EOF. This will
            //be a partially filled array of 128 elements.
            int sum2 = fileLength - sum;
            buf2 = new byte[sum2];
            for(int i = 0; i < sum2; i++)
            {
              buf2[i] = input.readByte();
            }
            fileOut.write(buf2);

            //CLEAN UP!
            //There is more data coming as a remainder of (128 elements
- sum2)
            //Therefore, we'll collect it and do nothing with it.
            //The data is empty anyway.
            int rem = 128 - sum2;
            byte[] b = new byte[rem];
            for(int i = 0; i < rem; i++)
            {
              b[i] = input.readByte();
            }

          }
          else if(fileLength <= 128)//file is less than 128 bytes
          {
             buf = new byte[fileLength];
             input.read(buf);
             fileOut.write(buf);
          }
          else //the file is over 1Megabyte, increase array size by
256bytes
          {
            buf = new byte[256];
            //Calculate the number of full arrays
            //are needed to receive the majority of the data.
            int x = fileLength / 256;
            int sum = x * 256;
            //load up the array

            for(int i = 256; i <= sum; i = i + 256)
            {
              input.read(buf);
              fileOut.write(buf);
            }
            //Now, calculate the remaining number of bytes
            //required to receive the data to EOF.
            int sum2 = fileLength - sum;
            buf2 = new byte[sum2];
            for(int i = 0; i < sum2; i++)
            {
              buf2[i] = input.readByte();
            }
            fileOut.write(buf2);

            //CLEAN UP!
            //There is more data coming as a remainder of (256 elements
- sum2)
            //Therefore, we'll collect it and do nothing with it.
            //The data is empty anyway.
            int rem = 256 - sum2;
            byte[] b = new byte[rem];
            for(int i = 0; i < rem; i++)
            {
              b[i] = input.readByte();
            }
          }
          //Close the stream
          if(fileOut != null)
            fileOut.close();
          //System.out.println("Closed stream");
          output.writeInt(session.UPLOAD_SUCCESSFUL);
          output.flush();
      }
    }
    catch(SecurityException se)
    {
      if(DirAdmin.hasInterface)
        myServer.display("SecurityException occurred in method
ReceiveFile");
      else
        System.out.println("SecurityException occurred in method
ReceiveFile");
      try
      {
        output.writeInt(session.UPLOAD_UNSUCCESSFUL_SECURITY_EXCEPTION);
        //Close the stream
        if(fileOut != null)
          fileOut.close();
        output.flush();
      }
      catch(IOException ioe){}
    }
    catch(IOException ioe)
    {  ioe.printStackTrace();
       try
       {
         output.writeInt(session.UPLOAD_UNSUCCESSFUL);
         //Close the stream
         if(fileOut != null)
           fileOut.close();
         output.flush();
       }
       catch(IOException io){}
    }
    catch(Exception e)
    {   e.printStackTrace();
        try
        {
          output.writeInt(session.UPLOAD_UNSUCCESSFUL);
          //Close the stream
          if(fileOut != null)
            fileOut.close();
          output.flush();
        }
        catch(IOException io){}
     }
   }


************************************************

On the client side app:
The reference to "timer" in uploadFile() is an instance of the
request_workerThread class.


 private void uploadFile(String FileToPathOnServer, String
FileFromLocalDrive)
  {
    File myFile = null;
    //Show the connection icon on the interface
    ref.showConnectionIcon();

    //Tell the server where to put the uploaded file
    try
    {
      output.writeInt(PATH);
      if(ref.serverOS.equals("Windows"))
      {
        output.writeUTF(convertSingleBackwardSlash(FileToPathOnServer));
      }
      else
      {
        output.writeUTF(FileToPathOnServer);
      }
    }
    catch(IOException ioe){System.out.println("IOException\n");
ioe.printStackTrace();}

    if(getPlatform().equals("Windows"))
    {
      String temp = convertSingleBackwardSlash(FileFromLocalDrive);
      //System.out.println(temp);
      myFile = new File(temp);
    }
    else
    {
      myFile = new File(FileFromLocalDrive);
    }

    int fileLength = 0;
    fileLength = (int)myFile.length();
    //Begin transfering the file at byte level
    try
    {
       //tell the server we're uploading a file
       output.writeInt(UPLOAD_FILE);
       /*
         1 megabyte (M / MB) = 2^20 bytes = 1,048,576 bytes
         The file size can quite possibly be quite large.
         So, we're going to set the array size to 128 elements
         for files less than 1 Mbyte and keep sending data
         in chunks of 128 bytes.
         We'll dimension the array to 256 elements if the file
         is larger than 1 Mbyte.
       */
       output.writeInt(fileLength);
       //System.out.println("Output filelength = " + fileLength);

       if(fileLength < 1048576 && fileLength > 128)//if the file is less
than 1Mbyte
          buf = new byte[128];
       else if(fileLength <= 128)
          buf = new byte[fileLength];
       else
          buf = new byte[256];

       //Wait or the signal to proceed before attempting to upload the
data.
       //We do this by invoking a worker thread which checks the
"proceed" and
       //"WebSpaceFull" flags every five seconds.
       //See request_workerThread class.
       //System.out.println("Calling timer functions.");
       ref.textLabel.setText("uploading " + myFile.getName());
       timer.setOnlyOneFile(true);
       timer.setByteArrayReference(buf, fileLength, myFile);
       timer.setThreadSuspended(false);

    }
    catch(IOException ioe){System.out.println("IOException\n");
ioe.printStackTrace();}
    catch(Exception e)
    {
       System.out.println("Exception\n");
       e.printStackTrace();
       ref.jLabel1.setText("Status: Exception in method uploadFile - You
may have lost your connection.");
       enableMenus();
    }

  }

*************************************************

request_workerThread, client side:

 import java.io.*;

 
public class request_workerThread extends Thread
{
  private boolean threadSuspended = true; //default
  private int timeElapsed = 0;
  private request rq = null;
  private byte[] array;
  private int SizeOfFile = 0;
  private FileInputStream fin;
  private File myFile;
  private boolean onlyOneFile;
  private boolean SaveFileAs;


  public request_workerThread(request req)
  {
    //System.out.println("request_workerThread instantiated.");
    rq = req;
  }

  protected synchronized void setByteArrayReference(byte[] reference,
int fileSize, File ref)
  {
    array = reference;
    SizeOfFile = fileSize;
    myFile = ref;
  }

  protected synchronized void setByteArrayReference(byte[] reference,
boolean SavingNewFile)
  {
    array = reference;
    SaveFileAs = SavingNewFile;
  }

  protected void setOnlyOneFile(boolean value)
  {
    onlyOneFile = value;
  }

  /**
   * This class extends class Thread, therefore we must overwrite the
run() method.
   */
  public void run()
  {
     while(true)
      {
         if(threadSuspended)
         {
            synchronized(this)
            {
               try
               {
                  while(threadSuspended)
                  {
                     //System.out.println("threadSuspended = true");
                     wait();
                  }
               }
               catch(IllegalMonitorStateException imse){}
               catch(InterruptedException ie){}
            }
         }

         try
         {
            Thread.sleep(5000);  //run a 5 second timer
         }
         catch(InterruptedException e){}

         //Record time elapsed.
         timeElapsed = timeElapsed + 5;
         //System.out.println("Elapsed time = " +
String.valueOf(timeElapsed) + " seconds.");

         //Check to see if the either flag has been set.
         //If not, after thirty seconds then put this thread to sleep
again.
         //System.out.println("proceed = " + rq.proceed + " WebSpaceFull
= " + rq.WebSpaceFull);
         if(rq.proceed || rq.WebSpaceFull)
         {
            try
            {
              if(rq.proceed)
              {
                //uploading only one file
                if(onlyOneFile && SaveFileAs == false)
                {
                  //System.out.println("Sending only one file");
                  //Access the file on local drive and start reading
from it.
                  try
                  {
                    fin = new FileInputStream(myFile);
                  }
                  catch(FileNotFoundException
fnfe){System.out.println("FileNotFoundException\n");
fnfe.printStackTrace();}
                  rq.ref.jProgressBar1.setMinimum(0);
                  rq.ref.jProgressBar1.setMaximum(SizeOfFile);
                  rq.ref.jProgressBar1.setStringPainted(true);
                  rq.ref.jProgressBar1.setVisible(true);
                  rq.disableMenus();

                  int n;
                  int counter = 0;
                  //System.out.println("Start sending data");
                  if(SizeOfFile < 1048576 && SizeOfFile > 128)//file is
less than 1 Mbyte
                  {
                    while((n = fin.read(array)) >= 0)
                    {
                      rq.output.write(array);
                      rq.ref.jProgressBar1.setValue(counter += 128);
                    }
                    //System.out.println("Finished sending data");
                  }
                  else if(SizeOfFile <= 128)
                  {
                    fin.read(array);
                    rq.output.write(array);
                  }
                  else //file is greater than 1 Mbyte
                  {
                    while((n = fin.read(array)) >= 0)
                    {
                      rq.output.write(array);
                      rq.ref.jProgressBar1.setValue(counter += 256);
                    }
                    //System.out.println("Finished sending data");
                  }

                  //Close the stream
                  try
                  {
                    if(fin != null)
                      fin.close();
                    //System.out.println("Closed stream");
                  }
                  catch(IOException
ioe){System.out.println("IOException\n"); ioe.printStackTrace();}

                  rq.output.flush();
                  rq.ref.jProgressBar1.setVisible(false);
                  rq.enableMenus();
                  rq.proceed = false; //reset
                  timeElapsed = 0; //reset
                  setThreadSuspended(true); //reset

                }
                //we're just saving a new file
                else if(onlyOneFile && SaveFileAs)
                { //System.out.println("Sending file data to server.");
                  rq.output.write(array);
                  SaveFileAs = false; //reset
                  setThreadSuspended(true);
                }
                //uploading many files
                else
                {
                  //System.out.println("Sending more than one file");
                  //Access the file on local drive and start reading
from it.
                  try
                  {
                    fin = new FileInputStream(myFile);
                  }
                  catch(FileNotFoundException
fnfe){System.out.println("FileNotFoundException\n");
fnfe.printStackTrace();}
                  //System.out.println("Processing " +
myFile.getName());
                  rq.output.writeUTF(myFile.getName());
                  rq.ref.jProgressBar1.setMinimum(0);
                  rq.ref.jProgressBar1.setMaximum((int)myFile.length());
                  rq.ref.jProgressBar1.setStringPainted(true);
                  rq.ref.jProgressBar1.setVisible(true);

                  int n;
                  int counter = 0;
                  if(SizeOfFile < 1048576 && SizeOfFile > 128)//file is
less than 1 Mbyte
                  {
                    while((n = fin.read(array)) >= 0)
                    {
                      rq.output.write(array);
                      rq.ref.jProgressBar1.setValue(counter += 128);
                    }
                    //System.out.println("Finished sending data");
                  }
                  else if(SizeOfFile <= 128)
                  {
                    fin.read(array);
                    rq.output.write(array);
                  }
                  else //file is greater than 1 Mbyte
                  {
                    while((n = fin.read(array)) >= 0)
                    {
                      rq.output.write(array);
                      rq.ref.jProgressBar1.setValue(counter += 256);
                    }
                    //System.out.println("Finished sending data");
                  }
                  //Close the stream
                  try
                  {
                    if(fin != null)
                      fin.close();
                    //System.out.println("Closed stream");
                  }
                  catch(IOException
ioe){System.out.println("IOException\n"); ioe.printStackTrace();}
                  rq.ref.jProgressBar1.setValue(0);
                  rq.output.flush();
                  rq.proceed = false; //reset
                  timeElapsed = 0; //reset
                  setThreadSuspended(true); //reset
                  rq.sendNextFile = true; //reset
                }
              }
              else if(rq.WebSpaceFull)
              {
                //Don't send any data!
                //System.out.println("received signal to abort upload");
                //We need to set flag: sendNextFile = true
                //if processing multiple files for upload so that we can
                //exit out of both the while and for loops in
rq.uploadFiles().
                //uploadfiles() will also attempt to reset flag:
WebSpaceFull = false.

                rq.sendNextFile = true;//reset
                rq.output.flush();
                timeElapsed = 0; //reset
                rq.WebSpaceFull = false; //reset
                setThreadSuspended(true); //reset
              }
            }
            catch(Exception e)
            {
              try
              {
                if(fin != null)
                  fin.close();
              }
              catch(IOException ioe){}
              e.printStackTrace();
            }
         }
         if(timeElapsed == 30)
         {
            timeElapsed = 0; //reset
            rq.ref.imageLabel.setIcon(rq.ref.blank);
            rq.ref.textLabel.setText("");
            rq.ref.jLabel1.setText("Status: Unable to complete File
Upload or File Save. Please try again.");
            rq.enableMenus();
            //System.out.println("Time elapsed is 30
seconds.\nrequest_workerThread is being reset to a suspended state.");
            /*There is a possibility that we'll have a time lapse of 30
seconds
             *and we're trying to upload multiple files. In this case,
since
             *request.uploadFiles() has a while loop in it, we need to
break
             *out of the while loop and the for loop. Therefore, we'll
set
             *both flags to true: WebSpaceFull and sendNextFile.
             */

             rq.WebSpaceFull = true;//reset
             rq.sendNextFile = true;//reset


            setThreadSuspended(true);
         }
      }
  }

  /**
   * Find out the state of threadSuspended.
   */
  public boolean getThreadSuspended()
  {
      return threadSuspended;
  }

  /**
   * Set the value of threadSuspended.
   * @param A boolean value.
   */
  public synchronized void setThreadSuspended(boolean value)
  {  //System.out.println("threadSuspended being set to " +
String.valueOf(value));
     threadSuspended = value;
     if(value == false)//we want to wake up this thread
        notify();
  }
}
Technical Details
Message-ID: <3D279904.389F6319@hfx.eastlink.ca>
Path: arc1-ca.pugleaf.net!archiver1.google.com!news1.google.com!sn-xit-02!sn-xit-06!supernews.com!iad-peer.news.verio.net!news.verio.net!news.maxwell.syr.edu!nntp-relay.ihug.net!newsfeeds.ihug.co.nz!ihug.co.nz!dnews
References: <3D20CC5D.B70473C9@hfx.eastlink.ca> <MPG.178acb912dba8f7998b79d@newsgroups.borland.com> <3D21C1CA.F2D953E2@hfx.eastlink.ca> <3d21bb44_1@dnews> <3D21EC78.228A5CD3@hfx.eastlink.ca> <3d21e07b_2@dnews> <3D226288.1C7C905@hfx.eastlink.ca> <3d22ec28_1@dnews> <3D233ADD.C365148D@hfx.eastlink.ca> <3d245dff_2@dnews> <3D24D4AC.1B1BFBBC@hfx.eastlink.ca> <3d24db4d_2@dnews> <3D25C73F.4D8D94E0@hfx.eastlink.ca> <3d25c545_1@dnews> <3D2619EB.4E07BAF3@hfx.eastlink.ca> <3d263c73_2@dnews>

Show Full Headers
Complete Article Headers
X-Google-Language: ENGLISH,ASCII-7-bit
X-Google-Thread: 1089d0,28eab50e784856ac
X-Google-Attributes: gid1089d0,public
X-Google-ArrivalTime: 2002-07-06 17:29:50 PST
Message-ID: <3D279904.389F6319@hfx.eastlink.ca>
Date: Sat, 06 Jul 2002 21:27:32 -0400
From: Alan Shiers <ashiers@hfx.eastlink.ca>
Organization: Mustard Seed Software
X-Mailer: Mozilla 4.78 [en] (Win98; U)
X-Accept-Language: en
MIME-Version: 1.0
Newsgroups: borland.public.jbuilder.java.api
Subject: Re: OutOfMemoryError
References: <3D20CC5D.B70473C9@hfx.eastlink.ca> <MPG.178acb912dba8f7998b79d@newsgroups.borland.com> <3D21C1CA.F2D953E2@hfx.eastlink.ca> <3d21bb44_1@dnews> <3D21EC78.228A5CD3@hfx.eastlink.ca> <3d21e07b_2@dnews> <3D226288.1C7C905@hfx.eastlink.ca> <3d22ec28_1@dnews> <3D233ADD.C365148D@hfx.eastlink.ca> <3d245dff_2@dnews> <3D24D4AC.1B1BFBBC@hfx.eastlink.ca> <3d24db4d_2@dnews> <3D25C73F.4D8D94E0@hfx.eastlink.ca> <3d25c545_1@dnews> <3D2619EB.4E07BAF3@hfx.eastlink.ca> <3d263c73_2@dnews>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
NNTP-Posting-Host: 24.222.101.2
X-Trace: dnews 1026001500 24.222.101.2 (6 Jul 2002 17:25:00 -0700)
Lines: 641
Path: arc1-ca.pugleaf.net!archiver1.google.com!news1.google.com!sn-xit-02!sn-xit-06!supernews.com!iad-peer.news.verio.net!news.verio.net!news.maxwell.syr.edu!nntp-relay.ihug.net!newsfeeds.ihug.co.nz!ihug.co.nz!dnews