Skip to main content

Writing huge string data to file in java, optimization options



I have a chat like desktop java swing app, where i keep getting String type data. Eventually the String variable keeps growing larger and larger.





1) Is it wise idea to keep the large variable in memory and only when the logging is finished save this to disk.





2) If not, then should i continue saving everytime i get a new string (of length about 30-40).





How should i go about optimizing such a desgin?


Comments

  1. I would use a BufferedWriter, like PrintWriter. This will buffer the data for you and write every 8 KB (actually every 8192 characters). If you want to write more often you can use flush() or a smaller buffer.

    PrintWriter pw = new PrintWriter("my.log");

    // will actually write to the OS, 5 times. (1000 * 40 / 8192)
    for(int i = 0; i < 1000; i++) {
    pw.printf("%39d%n", i); // a 40 character number.
    }

    pw.flush();


    or you can use

    pw.println(lineOfText);




    BTW: If you want to know what a really huge file looks like ;) This example writes an 8 TB file http://vanillajava.blogspot.com/2011/12/using-memory-mapped-file-for-huge.html

    ReplyDelete
  2. Perhaps you should use a StringBuilder. Append each new message to it, and at the end convert it to a string.

    For example,

    StringBuilder sb = new StringBuilder();
    // Do your code that continuously adds new messages/strings.
    sb.append(new_string);
    // Then once you are done...
    String result = sb.toString();


    If you were to have some string, say String message, and every time you got a new message/string you did message += new_string, it will eat up more memory.

    As suggested by Viruzzo, only save so much, then discard the earlier strings at some point. Don't hold on to every message forever.

    ReplyDelete

Post a Comment

Popular posts from this blog

Why is this Javascript much *slower* than its jQuery equivalent?

I have a HTML list of about 500 items and a "filter" box above it. I started by using jQuery to filter the list when I typed a letter (timing code added later): $('#filter').keyup( function() { var jqStart = (new Date).getTime(); var search = $(this).val().toLowerCase(); var $list = $('ul.ablist > li'); $list.each( function() { if ( $(this).text().toLowerCase().indexOf(search) === -1 ) $(this).hide(); else $(this).show(); } ); console.log('Time: ' + ((new Date).getTime() - jqStart)); } ); However, there was a couple of seconds delay after typing each letter (particularly the first letter). So I thought it may be slightly quicker if I used plain Javascript (I read recently that jQuery's each function is particularly slow). Here's my JS equivalent: document.getElementById('filter').addEventListener( 'keyup', function () { var jsStart = (new Date).getTime()...

Is it possible to have IF statement in an Echo statement in PHP

Thanks in advance. I did look at the other questions/answers that were similar and didn't find exactly what I was looking for. I'm trying to do this, am I on the right path? echo " <div id='tabs-".$match."'> <textarea id='".$match."' name='".$match."'>". if ($COLUMN_NAME === $match) { echo $FIELD_WITH_COLUMN_NAME; } else { } ."</textarea> <script type='text/javascript'> CKEDITOR.replace( '".$match."' ); </script> </div>"; I am getting the following error message in the browser: Parse error: syntax error, unexpected T_IF Please let me know if this is the right way to go about nesting an IF statement inside an echo. Thank you.