Questions tagged [dynamodb]

Amazon DynamoDB is a NoSQL Database featured as a part of the AWS.

Filter by
Sorted by
Tagged with
0 votes
0 answers
27 views

Updating attributes, including PK and SK. Move up to 10 Billion records to new table, while still capturing live changes

Looking to shorten the attributes for some of our tables in DynamoDB, including the PK and SK and am currently brainstorming on what would be the best way to achieve this. The largest table has 10 ...
cb1295's user avatar
  • 101
1 vote
0 answers
25 views

Best practice to store subitems

I am going to create my first Dynamodb database table. I have 3 options in my mind: 1. Create a long table where every subitem is a new row. Example: user John | item 1 | 2021 user John | item 2 | ...
L777's user avatar
  • 111
0 votes
0 answers
19 views

Access the same Entity via various attributes in DynamoDB

I'll begin by saying I am new to DynamoDB and the world of NoSQL in general. I have a patient entity and I am storing it in a DynamoDB table. My access patterns for the patient entity are: Get ...
J86's user avatar
  • 321
0 votes
1 answer
429 views

parse json into a relational database or use a NoSQL?

I am retrieving a JSON file from an API which size varies between 100kb to 7mb. The structure of the response is: I initially thought in storing the response in a relational database, with the ...
Manza's user avatar
  • 103
1 vote
0 answers
757 views

How to fully migrate data from dynamodb to postgresql

I have large data set inside dynamodb and want to move them all to postgresql and only can have 1 hour downtime. Whats the best tool or option to migrate and map all of my data from dynamo to postgres ...
Mj Ebrahimzadeh's user avatar
0 votes
0 answers
35 views

Does it make sense to separate records in a document DB (e.g. DynamoDB) for better query performance?

I've recently started working with DynamoDB, and most of my experience is in traditional data warehousing (e.g. sql server). One common pattern which I saw when working with SQL server was that we ...
Anton's user avatar
  • 101
0 votes
2 answers
3k views

How do I search by attribute in DynamoDB?

I inherited a site that uses DynamoDB for its database, which I know nothing about unfortunately, and I'm trying to change a user's email for them. There appears to be a users table in DynamoDB, but ...
kloddant's user avatar
  • 111
0 votes
1 answer
16 views

Help needed with several engines use case [closed]

We are developing an app, aprox 50k RPM read, 1k RPM write, it ask via key and get a JSON. The search is always made by key. I'm inclined to use one MySQL 8 table with a Id field and JSON field with ...
Alejandro's user avatar
  • 113
1 vote
3 answers
614 views

SQL vs NoSQL: Fetching All records in DynamoDB vs SQL Database

I am designing database for a project which requires frequent querying of all the records. The total number of records will be less than 500K. One of the databases I am considering is AWS DynamoDB ...
Oliver Blue's user avatar
1 vote
0 answers
110 views

Dynamo get items from last X days using composite key

I have the following sort key in a dynamo db table: YYYY-MM-DD#<USER_ID>#<CATEGORY_ID> Im trying to solve the following two access patterns: get all item for a user in the last x days ...
GWed's user avatar
  • 509
0 votes
1 answer
157 views

DynamoDB: How to model data that's shared and queried by all users?

I am very intrigued by DynamoDB, and it works incredibly well when I model the data for my main use case for my application. That being said, there is one specific use case that I can't wrap my head ...
Bradass's user avatar
1 vote
0 answers
44 views

How to implement correctly from SQL to NoSQL (DynamoDB)?

I am just new in NoSQL and DynamoDB. I try to moving from using AWS RDS to DynamoDB. I have table user which can have more than one of work experiences and can have more than one of interest (table ...
Ragam's user avatar
  • 11
-2 votes
1 answer
240 views

Can PostGIS or Oracle Spatial and Graph scale to hundreds of terabytes of data, or should I use a NoSQL option like DynamoDB? [closed]

I'm going to have a very large data set (a few hundred terabytes big eventually), but it is very well structured, relatively simple data involving lat, long points (which is why I want some GIS ...
Dylan's user avatar
  • 19
1 vote
0 answers
179 views

Most effective way of logging user query activity

We are logging and permanently storing all the queries performed about private persons in our database. Due to GDPR we need to be able to show all the entities who have requested personal data and the ...
Karl Märka's user avatar
1 vote
1 answer
72 views

Suitable NoSQL DB design for a fleet of IoT devices [closed]

This is my use case: 10,000 IoT devices Each IoT device sends its environmental data (temperature, pressure, etc.) every 30 seconds Data of each device will be kept for 24 hours The main requirement ...
madu's user avatar
  • 121
2 votes
1 answer
258 views

How does an encrypted DynamoDB (the same question applies to RDS as well) query work?

I am trying to understand whether the primary key is encrypted when I choose to encrypt at rest for AWS DynamoDB If the primary key is encrypted, how does a primary-key lookup get performed under the ...
chen's user avatar
  • 121
1 vote
1 answer
4k views

How to update DynamoDB item by searching on GSI primary key?

Table has these columns on DynamoDB: primaryKey sortKey GSI_primaryKey data And there is GSI added with primary key GSI_primaryKey and sort key on primaryKey How to update column data if I have ...
Lukas Liesis's user avatar
1 vote
0 answers
76 views

DynamoDB query by flag which tells if it's time to re-process the item

I have a table of stuff and some of it needs to be processed with cron daily or every minute. On SQL table I would add column last_time_processed and save timestamp in it. The query would take rows ...
Lukas Liesis's user avatar
3 votes
3 answers
224 views

Storing ~700 Tb in SQL Db

I am a newbie to Db's on a whole, and I came across a tutorial, wherein we needed to store ~700 Tb of data over a few years. It was mentioned that scaling SQL for such limits of data is not the best ...
Sanchay's user avatar
  • 131
12 votes
1 answer
8k views

When to use multiple tables in DynamoDB?

The DyanmoDB best practices make it clear that: You should maintain as few tables as possible in a DynamoDB application. Most well designed applications require only one table. I find it amusing ...
David Eyk's user avatar
  • 527
2 votes
0 answers
2k views

NoSql - Dynamo db - when do you need multiple tables?

I'm experienced in RDBMS, but new to NoSql. This aws dynamodb design advice says that "most well designed applications require only one table". I'm trying to wrap my head around that. I don't see a ...
Elroy Flynn's user avatar
4 votes
1 answer
2k views

How can I reuse a table name after deletion (in DynamoDB)?

I am working through a tutorial which requires creating a DynamoDB table with a specific name. On my first pass at creating the table I did something out sync with the tutorial, so I deleted the table ...
Lucas Gonze's user avatar
1 vote
0 answers
91 views

DynamoDB how to best perform != query?

Within the key-value, hash-table DynamoDB we can only query for partition keys and optionally for a sorting key. My Question is: If a Table A has items with primary partition attribute aId and I've ...
DrDirk's user avatar
  • 111
0 votes
1 answer
947 views

Insert into DynamoDB when data has different attributes

Is there any way dynamodb can handle dynamic attributes? Depending on the request, I need to store different web query parameters in the same table. What I want to do is like this, if I make a java ...
yaboong's user avatar
0 votes
1 answer
1k views

Porting a MySQL back-end to AWS DynamoDB

I'm setting up a back-end for a mobile application using Amazon Web Services. I'm fine with S3 and Lambda, but about using DynamoDB I have some problems. Before using AWS, I was developing the core of ...
mars's user avatar
  • 1
1 vote
0 answers
321 views

most efficient way to deal with real time gps data. S3 vs dynamoDB vs postgres

I have an application that is very write heavy. It saves GPS data every second. The way I handle it at the moment is to store the data locally and upload it to s3 every 5 minutes. The database (...
Chris's user avatar
  • 129
4 votes
1 answer
595 views

How should I use DynamoDB table

I have a list of items. Each item is attached to several urls and each url has a score number. So I have something like this : How should I organise this in dynamoDB? How should I format my Json file ...
Simon Breton's user avatar
2 votes
1 answer
465 views

Best way to store a list of numbers where individual numbers can be deleted

I want to store a list of 50 or less nine digit numbers for each primary key in a DynamoDB. Usually I would be using a get request to retrieve the list for a specific primary key. Sometimes I would ...
Ben Giles's user avatar
1 vote
1 answer
9k views

NoSQL : Getting latest item from DynamoDB table

I'm implementing chat server currently, with dynamoDB. Here is keys from ChatData table Primary partition key : roomNo Primary sort key : seq Chat server, can created many instant rooms, ...
Minkyu Kim's user avatar
2 votes
1 answer
583 views

Which AWS database services are designed for storing logging records like log4j or log4perl?

I have been using the file system for log4j and log4perl records and want to start using a database to store the information, especially to lookup records organized by pid. We are using MySQL RDS for ...
David's user avatar
  • 23
11 votes
2 answers
31k views

DynamoDB - Multiple range keys

Is it possible to have multiple fields as range key? Say I have a table where each row is uniquely identified by <A,B,C> ------------------------------- A | B | C | D | E | ----...
Thiyagu's user avatar
  • 213
0 votes
1 answer
78 views

After modifying the throughput of a DynamoDB table, where can I see the new throughput value before it gets active?

After modifying the throughput of a table in DynamoDB, the "Modify Throughput" is greyed out: Where can I see the new throughput value before it gets active? Sometimes it takes a while for a ...
Franck Dernoncourt's user avatar
4 votes
2 answers
2k views

Why are some write requests throttled whereas the average consumed write capacity is below the provisioned write capacity?

I see the following graphs in one of my DynamoDB table: Between 2/15 9:30 and 2/15 10:00 some write requests were throttled while the average consumed write capacity was below the provisioned write ...
Franck Dernoncourt's user avatar
-1 votes
1 answer
130 views

History of the provisioned write capacity in DynamoDB beyond the last two weeks

Is there any way to see the history of the provisioned write capacity for a DynamoDB table beyond the last two weeks? When I go to the Amazon AWS console, select a DynamoDB table and look at the ...
Franck Dernoncourt's user avatar
7 votes
4 answers
34k views

Import CSV or JSON file into DynamoDB

I have 1000 CSV files. Each CSV file is between 1 and 500 MB and is formatted the same way (i.e. same column order). I have a header file for column headers, which match my DynamoDB table's column ...
Franck Dernoncourt's user avatar
3 votes
1 answer
1k views

Best way to store, query and update 300M rows of data

I'm struggling to find a solution (preferably DBaaS) that I can rely on for storing and querying some 300M rows of data (roughly 100GB). The data in question is pretty much numeric. There is also one ...
Milovan Zogovic's user avatar
4 votes
2 answers
4k views

Using AWS DynamoDB Geospatial index with Python

Amazon's AWS DynamoDB features Geospatial Indexing, which facilitates geo queries: Query Support: Box queries return items that fall within a pair of geo points that define a rectangle as ...
Adam Matan's user avatar
  • 11.5k
4 votes
2 answers
10k views

DynamoDB: searching with primary key or range key, which way is faster?

To those familiar with DynamoDB, I'm trying to make a table where items only have 2 entries. For example, . Most of the time I'll be searching the user_id with a facebook_id on hand. Amazon's ...
Xavier_Ex's user avatar
  • 143