Tell me more ×
Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.
up vote 0 down vote favorite

I want to search scanned text (comprised of a few gigabytes of text).

The problem with this text (which is why I cannot make an index of words) is that it was OCRed, so there could be (and are) spaces in the middle of words (so, for example, computer could be c o m pute r).

On the other hand, to shrink the text (remove all spaces so the text of the page is somethinglikethissentence and searching through like "%this%" is slow.

Does anyone have any advice how to optimize the database?

share|improve this question
What RDBMS? Is this SQL Server? – JNK♦ May 9 '12 at 18:06
@JNK either SQLite or MySQL – Shmuel Brin May 9 '12 at 18:06
@JNK though if there is a better for this kind of search (and is available for free on Linux), I can use it also. – Shmuel Brin May 9 '12 at 18:07
I'm sure there are many ways to handle this but they would all require some sort of novel approach. Solr/Lucene may be good. Here's a thread that I found regarding what seems to be within the general domain of your presenting issue: lucene.472066.n3.nabble.com/Cleaning-up-dirty-OCR-td474429.html – swasheck May 9 '12 at 18:11
And another link that points to the hopeful "failed document" lucidimagination.com/devzone/technical-articles/… – swasheck May 9 '12 at 18:13
show 2 more comments

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.