I need help - I can help - Creating large files very slow

Print Page | Close Window

Creating large files very slow

Printed From: Debenu Quick PDF Library - PDF SDK Community Forum
Category: For Users of the Library
Forum Name: I need help - I can help
Forum Description: Problems and solutions while programming with the Debenu Quick PDF Library and Debenu PDF Viewer SDK
URL: http://www.quickpdf.org/forum/forum_posts.asp?TID=813
Printed Date: 11 May 24 at 5:17PM
Software Version: Web Wiz Forums 11.01 - http://www.webwizforums.com

Topic: Creating large files very slow

Posted By: mrgcms
Subject: Creating large files very slow
Date Posted: 20 Nov 07 at 9:57PM

We use QuickPDF to generate laboratory reports. On average, these run about 500 pages, but recently we've had a few large projects that resulted in 3000+ page reports. Probably 2000 of these pages consist of embedded graphics.

The report generation seems to go by fairly quickly, considering the amount of data that needs to be pulled together and streamed over the network. However, when we issue the QP.SaveToFile(FFileName) it takes another 30-40 minutes to actually create the file. This is on a 4.2GHz Athlon 64 X2. I don't have access to the sources, so I don't know what happens in the SaveToFile method, but I thought that the PDF was assembled in memory and that the SaveToFile routine just copied that data out to disk. If this were the case, it should only take a couple of seconds at most (64MB file size). Can somebody point me to what may be happening here, and is there anything I can do to speed this up?

Thanks

Replies:

Posted By: Ingo
Date Posted: 21 Nov 07 at 4:34PM

Hi!

Memory and time is often a problem working on big pdf files with quickpdf. It seems to me that in many code parts memory won't resetted ...

I'm using something like this:

   QP := TiSEDQuickPDF.Create;
   try
       QP.UnlockKey('mycode');
       dafh := QP.DAOpenFile(edit,'');
       x    := QP.DAGetPageCount(dafh);
       STR := '';

       verztxt := edit + '.txt';
       AssignFile(cf,verztxt);
       Rewrite(cf);
       pc := 0;
       for i := 1 to x Do
       begin
          dapr := QP.DAFindPage(dafh,i);
          QP.CombineLayers;
          STR := QP.DAExtractPageText(dafh,dapr,3);

          WriteLn(cf,Trim(STR));

          pc := pc + 1;
          if ( pc = 10 ) Then
             begin
                pc := 0;
                QP.DACloseFile(dafh);
                QP.Free;
                QP := TiSEDQuickPDF.Create;
                QP.UnlockKey('mycode');
                dafh := QP.DAOpenFile(edit,'');
              end;
// . . .

This works fine on big files for textextraction - no problems anymore for me. After few pages i'm using "free" and then a new "creation" ... This needs only less time and avoid memory-exceptions for me.

Best regards,
Ingo

Posted By: marian_pascalau
Date Posted: 22 Nov 07 at 4:57AM

Hi Dave,

you should send a test sample to support(at)quickpdf.org.

Posted By: mrgcms
Date Posted: 22 Nov 07 at 1:53PM

Hi Marian,

Unfortunately, it's in the report generation that this occurs, after I've assembled the 3,000 pages in memory. I'll put together a test app that uses a single graphics page that's repeated (I don't think QuickPDF is 'intelligent' enough to share these) and I'll see if it's still slow. If so, I'll upload a test project.

It's Thanksgiving over her now, so it may be Monday before I go back into the office and put this together.

Thanks

Posted By: Michel_K17
Date Posted: 23 Nov 07 at 9:27AM

Things I would check:

RAM: How large if the final report (in MB) and how much memory is in your PC?. Might it be large enough that Windows has run out of room in it's RAM and having to rely on the PageFile as extra memory?

NETWORK: Is it significantly faster if you save on the local HD rather than over the network?

Happy Thanksgiving. Drove 12 hours on Wed so that I could do the family (in-laws) get-together last night... Ate too much. :)

Cheers!

-------------
Michel

Posted By: mrgcms
Date Posted: 04 Dec 07 at 1:33PM

Final report is only about 16MB

System has 1GB RAM and an Athlon 64 X2 4.2GHz system

I've just repeated with the file being saved to the local drive without any improvement in speed.

We're at end of year crunch right now, but I'm trying to find time to put together a test program that'll demonstrate this.