I need help - I can help - text extraction

Print Page | Close Window

text extraction

Printed From: Debenu Quick PDF Library - PDF SDK Community Forum
Category: For Users of the Library
Forum Name: I need help - I can help
Forum Description: Problems and solutions while programming with the Debenu Quick PDF Library and Debenu PDF Viewer SDK
URL: http://www.quickpdf.org/forum/forum_posts.asp?TID=3974
Printed Date: 02 May 25 at 4:10AM
Software Version: Web Wiz Forums 11.01 - http://www.webwizforums.com

Topic: text extraction

Posted By: BAULOG
Subject: text extraction
Date Posted: 22 Feb 22 at 1:33PM

Hi,

at first, hello and greetings from Germany.

We have been using the Quick PDF library for many years and have solved all tasks well. But now I have a problem for which I cannot find a solution.

The problem is text extraction.

With TTF fonts I get with the function

GetTextBlockText(_TBID, _TBCounter))

I always get hieroglyphics like

T㠀䂎 쁰䀀䂏㴷砄␋Ȩ䁓T쳤視﫸〶�⻈ă徆㙺ăă抙㙺ăe㠀䂎 쁰䀀䂏㴷砄␋Ȩ䁓䂀䂀eࣘ〨튴밁젼視﫸〶�⻈ă徆㙺ăă抙㙺ăă抪㙺ăă搮㙺ăꚐ㯐Ꙡ㯐x㠀䂎 쁰䀀䂏㿰㴷砄␋Ȩ䁓䂎䁼䂎x㯘쳤밁﫸〶⻍ă徆㙺ăă抙㙺ăă抪㙺ăă搮㙺ăꚐ㯐Ꙡ㯐�⻈⻍t㠀䂎 쁰䀀䂏㿰㴷砄␋Ȩ䁓䂎䁼␀䂖t㯘튴밁꼤륫﫸〶⻍ă徆㙺ăă抙

If I get the whole text of the page with

GetPageText(0)

I get the correct text

"Text"

Can anyone help me?

regards

Peter

Replies:

Posted By: Ingo
Date Posted: 22 Feb 22 at 8:48PM

Hi Peter :)

It looks as if a Decrypt is missing after load and before extraction?
BTW: GetTextBlockText needs an ExtractPageTextBlocks first - what's the result of this function? Perhaps a longer, relevant code snippet makes it easier to say more...

Cheers and welcome here,
Ingo (living near Bremen ;-)

-------------
Cheers,
Ingo

Posted By: Ingo
Date Posted: 22 Feb 22 at 8:49PM

aditional hint:
A CombineContentStream and NormalizePage can help as well...

-------------
Cheers,
Ingo

Posted By: BAULOG
Date Posted: 23 Feb 22 at 8:02AM

Hi Ingo,

thank you very much for the quick answer. Should we stick with English, as perhaps other users should also benefit from it?

Here the code snippet.

I have omitted the variable declarations (all started with "_").

PDFLib is the reference to the Debenu dll.

If FileOpen(_File,_ID)

_PageCount = PDFLib.PageCount()

For _PageCounter = 1 To _ToPage

_PCounter += 1

_TextCounter = 0

PDFLib.SelectPage(_PageCounter)

PDFLib.NormalizePage(0)

_TBID = PDFLib.ExtractPageTextBlocks(3)

_TBCount = PDFLib.GetTextBlockCount(_TBID)

For _TBCounter = 1 To _TBCount

_TextList.add(PDFLib.GetTextBlockText(_TBID, _TBCounter))

FileClose(_ID)

End If

The PDF file was generated from a CAD application. All TTF texts (Arial etc.) cannot be read correctly. The SHX fonts (CAD specific) are read correctly.

I would like to attach the pdf file. But I can't find a way to do it.

many Thanks

Peter

Posted By: Ingo
Date Posted: 27 Feb 22 at 8:20PM

Hi Peter,

perhaps the pdf is encrypted?
I don't see a decryption in your code...
There are free online hoster offering a bit space combined with advertisement.
You can use it with your sample-pdfs and post the link here.

-------------
Cheers,
Ingo

Posted By: tfrost
Date Posted: 28 Feb 22 at 2:04PM

Your "hieroglyphics" look to me like 'mojibake', which is the name for Japanese (or Chinese) characters appearing when you mix up ASCII and DBCS. Have you checked in a debugger that you are not accidentally assigning the wrong character size somewhere?

Posted By: BAULOG
Date Posted: 08 Mar 22 at 1:04PM

Hi,

at the following link you find a sample file with 4 lines text. Each text has a different font.

I only get the first line as an readable text.

http://cloud.baulog.de/index.php/s/4cSjGXdLXfcZqeJ" rel="nofollow - https://cloud.baulog.de/index.php/s/4cSjGXdLXfcZqeJ

Maybe someone else has an idea.

Thanks for your help.

Peter

Posted By: tfrost
Date Posted: 08 Mar 22 at 3:35PM

I think it's a bug (which I can reproduce) in GetTextBlockText. After each valid character in the 130 characters returned for each (5-character) line of your text, there is a block of garbage 8-bit characters which form the mojibake string. This happens in every text line except your first one. Dropping all characters not in 0..9,a..z,A..Z reveals the correct results.

It looks as if GetPageText(0) sanitizes the output somehow, to get rid of all the garbage. But it is hard to tell from the source where it does this, because the primary filter in this function is the area covered, and the filtering on content must then happen below this. My (ancient) copy of Debenu PDF Tools Pro must use this function, because it also displays all the text correctly when extracting.

Interestingly, I opened your PDF in Affinity Publisher (which can edit PDFs), and found that Text1 completely disappears, whereas the other three are clear and editable, in substituted fonts Consolas, Tahoma and Arial respectively.

Posted By: Ingo
Date Posted: 08 Mar 22 at 10:14PM

I've got the same described issues with normal
GetPageText with option 7 and option 3!

My sample looks like this:

--- page 1 from 1 ---

"Arial";000000;0;234/118;"Text1"

"AAAAAA+ISOCPEUR";000000;105;234/241;"T㠀䂎 쁰䀀䂏㎀ӷ툠彯切䁚T�ĕὈˑ糌˔ꯣ}꼹}e㠀䂎 쁰䀀䂏㎀ӷ툠彯切䁚䂀䂀eꄸӲ�ĕ艔ĕὈˑ糌˔ꯣ}꼹}꽊}냎}궠Ӯ군Ӯx㠀䂎 쁰䀀䂏㎀ӷ툠彯切䁚　䁼㠀䂎x鏨Ӻ萴ĉ분đὈˑ糌˔ꯣ}꼹}꽊}냎}궠Ӯ군Ӯ῜ˑt㠀䂎 쁰䀀䂏㎀ӷ툠彯切䁚　䁼⠀䂖t׭脴Č戌ĎὈˑ糌˔ꯣ}꼹}꽊}냎}궠Ӯ군Ӯ῜ˑ2㠀䂎 쁰䀀䂏㎀ӷ툠彯切䁚　䁸"

"AAAAAA+Tahoma";000000;93;234/369;"T⠀䂐　쁺䀀䂏㎀ӷ氠穸咥䁗T脴Č׭耼˔ꯣ}꼹}e⠀䂐　쁺䀀䂏㎀ӷ氠穸咥䁗䀀䂂䀀䂂eꄸӲ脴Č矌Č׭耼˔ꯣ}꼹}꽊}냎}궠Ӯ군Ӯx⠀䂐　쁺䀀䂏㎀ӷ氠穸咥䁗瀀䂀堀䂑x鏨Ӻ脴Č鈌Ď׭耼˔ꯣ}꼹}꽊}냎}궠Ӯ군ӮὌˑt⠀䂐　쁺䀀䂏㎀ӷ氠穸咥䁗䁾᐀䂙t׭脴Č戌Ď׭耼˔ꯣ}꼹}꽊}냎}궠Ӯ군ӮὌˑ3⠀䂐　쁺䀀䂏㎀ӷ氠穸咥䁗䁴"

"AAAAAA+ArialMT";000000;95;234/497;"T䀀䂐倀쁴䀀䂏㎀ӷ鄀ﭾ뀺䁗T矌ĕ䋸ˑ׭갦}꼹}e䀀䂐倀쁴䀀䂏㿰㎀ӷ鄀ﭾ뀺䁗က䂃᠀䂃က䂃eꄸӲ职ĕ䋸ˑ׭갦}꼹}꽊}냎}궠Ӯ군Ӯx䀀䂐倀쁴䀀䂏㿰㎀ӷ鄀ﭾ뀺䁗က䂃怀䂁㠀䂒x鏨Ӻ脴Č职ĕ䋸ˑ׭갦}꼹}꽊}냎}궠Ӯ군Ӯ׭׭t䀀䂐倀쁴䀀䂏뿰㎀ӷ鄀ﭾ뀺䁗ఀ䂚䀀䁿ఀ䂚tৈ׭脴Č䋸ˑ׭갦}꼹}꽊}냎}궠Ӯ군Ӯ׭׭4䀀䂐倀쁴䀀䂏㿰㎀ӷ鄀ﭾ뀺䁗怀䂞怀䁱 "

-------------
Cheers,
Ingo

Posted By: tfrost
Date Posted: 09 Mar 22 at 12:10PM

Just to save others the trouble, I found that the DA versions of these functions produce the same garbage output, although the source code for the top-level functions is completely different.

The winsoft.sk PDFIUM wrapper has an extract text sample which extracts all four of your examples correctly. It is available for Delphi or .Net but I have only tested the former. I feel able to mention it here because of the QPDF EOL announcement and the slim chance of this being improved in QPDF.