I have a table of stuff and some of it needs to be processed with cron daily or every minute. On SQL table I would add column last_time_processed and save timestamp in it. The query would take rows like

SELECT * FROM table WHERE last_time_processed < 123456 and would add an index on this column.

How to do the same thing on DynamoDB? I have to select primary key to make a query and primary keys are different for different rows and I need to select all rows where last_time_processed < 1234.

The scan would make a job but it would read the entire table, which will eat a lot of RCU.

Thinking about another way to add an extra column which would have fake-primary-key-x and x would be anything between 1..10 so, in theory, it would spread GSI to 10 nodes and bypass 10GB limit/node. And then query on each primary key + sort by last_time_processed but it would require several queries to make full select.

Any ideas?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.