1

I have an application that is very write heavy. It saves GPS data every second. The way I handle it at the moment is to store the data locally and upload it to s3 every 5 minutes. The database (dynamoDB) then only contains the metadata about the gps file.

Now I would like to get all the gps data into the db rather than just the metadata. I thought about a aws lambda function which automatically reads the gps file and saves the entries to the db, but this has one big problem.

DynamoDB is priced by write capacity (i.e. about 5$ per month for every 10 writes/s). So I either have to pay a lot of money to get the necessary capacity (5min = 300 entries/writes, multiplied by the amount of simultaneously uploaded files) or throttle the db writes to slowly process all the entries (this is not really an option since I need the file contents to get in the DB close to realtime).

Therefore I consider switching to AWS RDS postgresql since it does not charge per request and also allows multi item inserts. Does anybody have experience regarding the performance on a micro instance? Can it handle potentially hundreds of incoming multi row insert statements (about 300-1000 items each).

Another short question. I use csv to store the gps data since it uses much less space than json and therefore saves bandwith. What does the result of a select statement look like? Is it something like:

firstname,lastname
steve,jobs
michael,jackson

or rather:

firstname: steve, lastname: jobs
firstname: michael, lastname: jackson

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.