0

I am new to databases so please forgive my mistakes. Here's the scenario. I use the web for my data source. Sometimes, like, for a keyword, what my crawler do is crawl same keyword repeatedly (not the crawler's fault).

I don't want to include that particular keyword again in my database. I know of primary key and also know that no two primary keys can be same.

So should I make my keyword primary key to avoid duplicate rows or is there some other methods to deal with it??

The DB is dynamic because there is very little manual interference in the crawl.

Note: I am using Apache Cassandra.

3
  • Thankyou for replying. I'll check it.
    – Aman Rawat
    May 26, 2022 at 14:00
  • 1
    Hi, and welcome to dba.se! What database server are you using? Always include it in the tags and/or the body of the question.
    – Vérace
    May 26, 2022 at 16:22
  • Thankyou for reply. I have updated the question. BTW database I am using Apache Cassandra
    – Aman Rawat
    May 27, 2022 at 15:20

1 Answer 1

-2

Having Primary Keys or Unique constraints are great options but you would need additional error handling for this and the cost of overhead might not be worth it for your needs.

Another option would be an insert like this:

IF NOT EXISTS (SELECT 1 FROM Words WHERE Word = @newWord) BEGIN INSERT INTO Words (Word) VALUES (@newWord) END

That should do what you're hoping for and remain lightweight. Other option would be an on INSERT trigger to check same thing if you don't want to handle in your app logic and want it on DB side.

3
  • 2
    So this is "lightweight" compared to a unique constraint? You still have to maintain a (non-unique) index, do an extra roundtrip for the query, and risk a race condition that would cause duplicates anyway.
    – mustaccio
    May 26, 2022 at 16:05
  • "lightweight" was in reference to handling and throwing errors when hitting into constraints when you're inserting to a table and throwing an error each time you try to insert an existing word. Which if you're scraping a website could be 1000s of duplicate words per page. Do I think having a unique constraint is the better option if you were to build in error handling to your application or SQL scripts? Of course. I said they're great options but if he is just trying to get the task done quick and dirty like I said "might not be worth it for your needs" I don't know what he needs it for.
    – Ron Zimmer
    May 26, 2022 at 18:58
  • Nothing prevents you from implementing exception handling on the database side, by various means available in different DBMSes. So you could still use a unique constraint, as dog intended, and avoid having to handle the potential constraint violation on the application side (though you still have to handle all the other exceptions there, don't you).
    – mustaccio
    May 26, 2022 at 19:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.