Questions tagged [aws-aurora]
Managed cloud database service from Amazon Web Services, available for Postgres or MySQL.
40
questions
0
votes
0answers
6 views
AWS DMS: Table Mapping?
We are moving from SQL Server to AuroraDB (mySQL). Both our source database (SQL Server) and our target (AuroraDB) are on AWS RDS so we are naturally using DMS for the migration.
We make heavy use ...
0
votes
0answers
13 views
Difference between Aurora Global database and Aurora regional database with multiple read replicas?
I am planning to migrate my 300GB RDS PostgreSql database to Aurora MySql using SCT and DMS. RDS Postgresql is in seven regions in current setup. Datapipelines are used to ingest data in these ...
2
votes
0answers
23 views
RAISE NOTICE doesn't work as expected while using within Dynamic SQL block
I have a small piece of code to get all sequences and the current and maximum value of all sequences in a database. I have tried with recursive and some other alternatives through plain sqls, however ...
1
vote
1answer
88 views
AWS Aurora running out of freeable memory and crashing
We have an issue with our AWS Aurora MySQL instance running out of freeable memory and crashing as a result. AWS's response was to upgrade to a bigger instance, but I don't feel that will necessarily ...
0
votes
1answer
36 views
Hints on bulk loading 3.6 billion rows to InnoDB on Aurora MySQL
everyone!
I'm struggling for a week to bulk load 3.6 billion rows to an InnoDB table on Aurora MySQL 5.6.10a. This table has one FK to a "Main" table and has 12 columns. The first 1.4 billion were ...
1
vote
1answer
68 views
AWS Aurora MySQL Serverless CPU utilization is growing for unknown reason
We have been using Aurora serverless databases in staging and production environments for a few months now, and find the CPU remains in the range 15-20% almost all the time. However in the past week ...
1
vote
1answer
42 views
Optimize Postgres query where a time range as well as IN values
I have a table that gets around 20 million new records a day for positions of vessels. I want to query for a couple of different vessels and get the records for any period. As I'm using this for ...
0
votes
0answers
84 views
Replacement to JSON_ARRAYAGG in MySQL v5.7.12
I have to group by some data from JSON fields, but how do I do that if I can't upgrade to 5.7.22? I can't do that because of requirement to use AWS Aurora which lags behind and only supports 5.7.12 ...
0
votes
0answers
279 views
AWS DMS Fails with “Command failed to load data with exit error code 1”
I am trying to Migrate my Postgresql RDS to Aurora Postgresql using DMS.
All my tasks are failing, and I am unable to understand what can be done to help the tasks progress.
The error from cloud ...
0
votes
0answers
78 views
AWS Aurora queries much slower in “sending data” phase than RDS instance
I've been using an RDS instance during the development phase of a project. This was a db.t2.micro instance, so not very powerful at all.
Today I have created a new Aurora (5.6) [serverless] database ...
0
votes
0answers
13 views
Can I use EBS data directory path as temp tablespace in ADS Aurora Postgresql?
I am having issue with RDS Aurora Postgresql's root storage (which is used for Temp files/tables and Logs) since its basically 2*PhysicalMemory. I thought of creating a new tablespace in data storage ...
0
votes
1answer
149 views
AWS Aurora MySQL Serverless CPU utilisation is above 10 even it is not processing any requests
We have Aurora MySQL serverless DB with 2 capacity units. I didn't understand why the CPU utilisation is always above 10 even when it not processing any queries. As a result we are paying more bills ...
1
vote
0answers
161 views
Predicting AWS RDS MySQL Instance Performance
I have a small MySQL db running on AWS RDS t2.small instance.
So far, the CPU usage has been very predictable based on the traffic from my app.
Example:
1000 concurrent users = 15% CPU
2000 ...
0
votes
1answer
90 views
Speed up UPDATE on large table
We are migrating our ids from INTEGER to BIGINT in some of our tables. The first step for us is to create a new temporary column id2 that has the correct type, and then we want to migrate the existing ...
0
votes
1answer
49 views
Find queries causing a lot of read IO
I'm trying to find the cause of a higher amount of VolumeReadIOPs in our Aurora Postgres instance on AWS. This is what the past 6 weeks look like:
I found the cause of the largest period (02/27 to 03/...
0
votes
0answers
27 views
Stop connections to Aurora PostgreSQL database
I am working towards upgrading my Aurora PostgreSQL database from 1.3 (pg 9.6) to 2.2 (10.6). Since there is no major upgrade supported, I am planning to take a pg_dump and do a restore on the latest ...
0
votes
0answers
216 views
Load From S3 MySQL Error Code 2013 — AWS Aurora
I am attempting to load S3 files to an Aurora MySQL DB Cluster using the below AWS documentation:
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Integrating.LoadFromS3.html
...
5
votes
0answers
132 views
Database design for ~ 2.7B rows per year / 26.000 data points every five minutes
The current situation
We're currently looking into a new product that will send device data back to us to interpret.
These are the numbers we're looking at:
Devices will most likely send data every ...
0
votes
0answers
34 views
How to change the Aurora MySQL instance role
I created a Aurora MySQL cluster a few weeks ago.
There are two instances in the cluster.
One has a writer role and the other has a reader role.
AS-IS
+----------------...
1
vote
0answers
360 views
Got an error reading communication packets on writer instance only
So I am connecting with aurora MySQL 5.7.12-log with Django. Below (1) is the query auto-generated by Django when I hit runserver.
SELECT `django_migrations`.`app`, `django_migrations`.`name` FROM `...
2
votes
1answer
662 views
Postgres RDS: is there a way to use LDAP authentication?
We have a client interested in using LDAP (specifically Active Directory) for user authentication on a PostgreSQL AWS RDS (Aurora, specifically) database.
Anybody doing this successfully?
0
votes
1answer
27 views
Reducing latency in read operations
Table size 45 million rows
Schema: item name, categories and subcategories, order quantity and price.
What is the fastest way to put each item into an appropriate bucket?
For example: price range ...
1
vote
1answer
63 views
Optimizing query performance with aggregation tables and partitioning
Our setup is a bit complex, so I will try to simplify what our tables look like but still try to describe the real problem:
We have a large table (items) with about 500 million rows (grows with ~1....
0
votes
0answers
9 views
Benchmark for a rowlock in mysql
I am trying to benchmark a sql transaction with a select..for update statement which uses an exclusive lock on the row and then insert a row into another table as shown below.
START TRANSACTION;
...
0
votes
1answer
208 views
Dump roles on AWS without pg_dumpall -g access (postgresql 9.6.8)?
We have a system on AWS RDS Aurora, PostgreSQL v 9.6.8 option.
We are trying to move its roles to a new encrypted 10.5 instance, but we don't have superuser access that's required to do pg_dumpall -g ...
3
votes
0answers
350 views
AWS Aurora Database Corruption
I'm running Aurora Postgres and noticed that metric Maximum Used Transaction IDs is increasing a lot. Trying to manually vacuum the tables that's was not automatically vacuumed lead me to the ...
1
vote
1answer
420 views
Downgrading from AWS Aurora MySQL 5.7 to AWS Aurora MySQL 5.6
We are trying to downgrade from AWS Aurora MySQL 5.7 to AWS Aurora MySQL 5.6 due to some issues. But AWS RDS doesn't allow Aurora MySQL 5.7 snapshot restore to Aurora MySQL 5.6 instance/cluster. Does ...
0
votes
0answers
47 views
Aurora MySQL 5.7 DDL statements hangs intermittently
We are using AWS Aurora MySQL and we recently moved to MySQL 5.7 version. After moving to this version, we started seeing issues with DDL statements. The processes which are running these statements ...
2
votes
1answer
47 views
Seeing AWS PostgreSQL-on-Aurora snapshot sizes of 0 GB - is this normal for a db of ~800mb?
While working on some code to manage our manual snapshots, I noticed that a number of recent AWS PostgreSQL-on-Aurora snapshots show a storage size of 0 GB (older ones are at 1 gb). I thought it might ...
3
votes
1answer
1k views
ERROR: improper qualified name (too many dotted names) when trying ALTER FUNCTION
Working with Amazon Aurora Postgres Version 2.0, which is basically PostgreSQL 10.4 with a modified storage layer.
Trying to restore a dump that contains multiple lines like:
ALTER FUNCTION ...
0
votes
0answers
135 views
Finding Aurora functions (PostgreSQL)
On Amazon Aurora in the PostgreSQL (10) flavor there is a function names aurora_replica_status() documented. I can use the function and look at its definition:
postgres=> select * from ...
-1
votes
1answer
59 views
What is a beachhead? (replication / sharding)
I quote the following from this blog post from AWS:
Because you’re running on MySQL, you can use one of the methods listed in the Aurora documentation to establish a beachhead in Aurora.
What ...
1
vote
1answer
255 views
migrating from a local PostgreSQL database to Amazon Aurora
I have a local PostgreSQL database and I want to migrate it to Amazon Aurora, how do I do that? I found a documentation on how to migrate from RDS PostgreSQL database, however, mine is local and I ...
0
votes
1answer
527 views
Error log flooding with aborted connections (Aurora 2.03 - MySQL 5.7.12)
Here's some contextual information: I have a multi-tenant, WordPress MU installation running on Elastic Beanstalk, scaled up to 2 instances minimum. These are connected to the RDS database and are ...
0
votes
1answer
31 views
How do I debug max_allowed_packet going to -1?
I am using Amazon's MySQL Aurora and using JDBI on the client side. Every once in a while, the connection goes bad and I get errors like this -
com.mysql.jdbc.PacketTooBigException: Packet for query ...
1
vote
1answer
661 views
Is Postgres sharding available on Aurora (AWS)
I need to shard and/or partition my largeish Postgres db tables. The Postgres partitioning functionality seems crazy heavyweight (in terms of DDL). I see talk from <=2015 about pg_shard, but am ...
2
votes
0answers
34 views
MySQL Queries Truncate Causing Connection Spikes
We have a MySQL Master Instance & Read (failover only) on AWS. MySQL 5.6 Aurora running Magento. Performance is great, and better than standard MySQL (we have high traffic site so see the benefits)...
1
vote
1answer
337 views
Are read replicas heavy reading in affect masterDB?
We have a very heavy load in Read side with large DB tables in 2 hours per day that affect to Db so as solutions I have created read replica for it.
If application fetches large heavy data on read ...
-2
votes
1answer
59 views
Which database is suitable for serving big timeseries metrics with rollups? [closed]
I have big mysql table containing daily metrics for large number of subjects. Here is the hypotetical schema:
day DATE
subject_id INT
metric1
metric2
metric3
What I want is to find top X subjects (...
1
vote
1answer
192 views
Can AWS Aurora High Availability allow different Schema on Replicas?
Our company is thinking of migrating from SQL Server to Amazon AuroraDB.
In SQL Server, there are AlwaysOn and Transactional Replication - two different things.
AlwaysOn is utilized more for ...
