Questions tagged [aws]

Amazon Web Services.

Filter by
Sorted by
Tagged with
0 votes
1 answer
39 views

New rows are locked by select on PostgreSQL replica

have a strange trouble with getting new rows on replica on PostgreSQL on AWS. I have master database on AWS with constant flow inserts/updates not with big rate. And these changes are replicated ...
1 vote
0 answers
22 views

Postgres: copy from 2 different DBs for identical tables

I have two identical DBs, under the same port and host. (they both postgres DBs and on AWS) I want to create a script\write any query (even though it is not possible to create a query that use data ...
0 votes
0 answers
20 views

Is there a way to export AWS RDS postgres snapshot to on-premise server?

I am working towards exporting a snapshot of RDS Postgres to S3 and pull it to on-premise server to cater to a requirement where I can run the database locally. Has anyone tried this before? and can ...
0 votes
0 answers
18 views

Describe table from Athena data source in DataGrip fails

Whenever I attempt to Describe an AWS Athena table in DataGrip it fails. The error states that is is a permissions error but I am fairly certain it is not. In fact, I can describe tables using ...
2 votes
2 answers
114 views

Operational Error on MySQL AWS RDS Instance

The heading pretty much describes it all, here are some more information on the Amazon RDS I am using. Engine: MySQL Community 8.0.28 Class db.r6g.large Multi-AZ no. The data volume is very small, ...
  • 73
2 votes
1 answer
67 views

AWS to Azure Degraded Performance With "Hash Match (Aggregate) Step"/"Temporary Disk" TempDB Performance Issue

Ok, so I have a series of AWS VM's, mostly using the z1d.3xlarge server class with built in SQL Server 2019, these servers each contain a NVMe of various size. (Not surprisingly) I've placed my SQL ...
0 votes
0 answers
21 views

Where clause in which the value of column2 changes based on column1

So I have a table which has State1: id column1 column2 1 myboolvalue false 1 myecogvalue 0 1 myintvalue 1700 State2 id column1 column2 1 myboolvalue true 1 myecogvalue 0 1 myintvalue 1700 In ...
0 votes
0 answers
22 views

RDS Postgres v14.4 CPU utilization not going below 5% when idle

I have an RDS Postgres instance on a t3.small and am trying to drive down the CPU usage of my clients connecting to it (I have a few API services connected). I've stopped all my services connecting to ...
  • 101
0 votes
0 answers
23 views

How to setup replication between a new aws rds instance and an existing aws rds instance using rds_set_external_master

I have an existing aws rds master (Source Master) and replica instance (Source Replica). I have created a new aws rds instance (Target Instance). Goal: I would like to setup replication so the Target ...
  • 109
0 votes
0 answers
20 views

Does AWS RDS Oracle support SSL between Standby and Read Replicas?

Does AWS RDS Oracle support TLS 1.2 between cluster members e.g. Standby (Multi-AZ deployment) and Read Replicas? I cant seem to find any documentation to confirm this.
0 votes
0 answers
27 views

Finding distance between 2 geospatial location in aws neptune with opencypher query language

I am working with amazon neptune and using openCypher query language and doing some stuff using notebook. I have latitude and longitude of 2 points and I want to calculate the distance between these 2 ...
1 vote
0 answers
11 views

How to encrypt a production unencrypted RDS Aurora MySQL 8 cluster

I have a running RDS Aurora MySQL 8.0.23 cluster running in production. The database is unencrypted and I need to enable encryption for it. As far as I understand, this is not possible to do directly. ...
  • 111
1 vote
0 answers
38 views

Redshift User password change permission denied

Redshift user not able to change its own password getting below error: ALTER USER or_user WITH PASSWORD 'md5<hash>'; ERROR:permission denied Also tried granting 'ALTER USER' privileges using ...
  • 11
1 vote
0 answers
48 views

Restart a Running Sum after Static Threshold

I've been stuck trying to solve the following problem for some time: I essentially want to restart the rolling sum if the rolling sum exceeds 3600, else continue summing. I've sort of have it working ...
0 votes
0 answers
20 views

Can we use AWS SCT (Schema Conversion Tool) with AWS DMS for ETL and migrating data from MySQL to Amazon Aurora for MySQL?

I want to migrate data from on-premises MySQL/MySQL RDS to amazon Aurora for MySQL. While migrating, I need to convert schema as well as do some ETL to get required schema and data. In ETL I need to ...
1 vote
0 answers
20 views

Best practice to store subitems

I am going to create my first Dynamodb database table. I have 3 options in my mind: 1. Create a long table where every subitem is a new row. Example: user John | item 1 | 2021 user John | item 2 | ...
  • 111
3 votes
2 answers
151 views

Need to keep each binlog files to 24 hrs

We have a requirement that we need to keep each binlog files(like mysql-bin-changelog.025249, mysql-bin-changelog.025250, mysql-bin-changelog.025251....) of MYSQL RDS to minimum 24 hrs. However, i see ...
0 votes
0 answers
112 views

Getting "The provided token has expired" error message in AWS DMS

I have set up a DMS task with the following setup: RDS MYSQL as source endpoint and REDSHIFT as target endpoint with "full load and CDC". setup was working fine for couple of days then ...
1 vote
0 answers
62 views

Backup data from RDS to S3 and delete from RDS once copied to S3

We have a requirement werein we have to backup data from RDS(MYSQL and Postgres) to S3 incrementally and to delete the already copied data from RDS. I thought of using aws data-pipeline for this, ...
0 votes
0 answers
26 views

Need to create DMS task thru AWS CLI

I have to migrate from MYSQL RDS to REDSHIFT thru DMS. There are 400+ tables in my source MYSQL RDS instance. i have to create 4 DMS task with 100 tables each thru AWS CLI since thru console it will ...
0 votes
0 answers
24 views

Using AWS DMS for EC2 MariaDB instance to MariaDB RDS instance

After combing through AWS documentation, I am still unclear if DMS supports migration from EC2 instances running latest mariadb version to RDS mariadb instance?I see that MySQL is supported, but am I ...
0 votes
0 answers
119 views

Getting '.....successive recovery failures' error in DMS while using postgres read replica as source

I'm getting below error in AWS DMS while migrating from Postgres read replica to Redshift. I'm using full load only.The DMS task will run for few mins and some tables will be loaded to target and then ...
0 votes
0 answers
91 views

Getting ODBC general error in DMS while using postgres read replica as source

I'm getting below error in AWS DMS while migrating from Postgres read replica to Redshift. I'm using full load only.The DMS task will run for few mins and some tables will be loaded to target and then ...
0 votes
0 answers
196 views

How to connect PgAdmin4 to postgresql 9.5 database process running on AWS EC2 server

I have a postgres database running on a linux EC2 instance. I want to view the data inside the tables, and currently the only way I can do that is via command line on the host server by: sudo su ...
3 votes
1 answer
158 views

Getting Error 1236 (Could not find first log file name in binary log index file) reading binlog in AWS DMS

I have created two DMS task (full load + CDC) with RDS MYSQL 'Read' replica as source endpoint and REDSHIFT as target endpoint. For both the task source and target endpoint is same. One task with only ...
0 votes
0 answers
14 views

Secure way for a public website to post requests with an AWS rest api/database

I am quite new to using rest type apis, and I am currently trying to develop a public website that I want to be able to post data to a backend (possibly via an intermediate DynamoDB on AWS or similar ...
  • 101
3 votes
3 answers
53 views

How can I make a simple left join with null condition run faster?

I've got two very simple tables CREATE TABLE `track` ( `id` char(36) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL, `number` int NOT NULL DEFAULT '-1', `title` varchar(255) CHARACTER SET ...
  • 143
0 votes
1 answer
107 views

AWS RDS database uses more space after pg_dump/pg_restore

I've got an RDS Postgresql 12 instance that allocated 11,000,000 MB of io1 disk space, and there's 7,300,000 MB of Free Storage Space. That means the database is only using 11TB - 7.3TB = 3.7TB of ...
  • 559
0 votes
0 answers
53 views

Redshift from dc2 to ra3 experiences

I'm currently running a 3 x dc2.8xlarge Redshift server, it's all good but I'm getting out of space (7.7TB available). So I have two options to maintain the current price (approx 10k/month): 4 x ra3....
  • 103
1 vote
1 answer
93 views

How can I load a custom format dump from pg_dump into an RDS while ignoring constraint checks during the import?

I have a number of constraints: I have binary custom format dump from Heroku that I need to load via pg_restore. I need to load this onto an AWS RDS instance, where I don't have full superuser (only ...
1 vote
1 answer
276 views

How to disable binlog_format in AWS RDS?

I created RDS on AWS and created MySQL v5.7 instance. I tried to find the binlog_format settings in parameter group but it seems to me that there are only three possible settings - like the screenshot ...
  • 115
0 votes
0 answers
61 views

How can I speed up INSERT INTO ... SELECT ... ORDER BY queries?

I recently learned that it is advisable to have deterministic queries when using a replication instance. Therefore, we started adding ORDER BY to our INSERT INTO (...) SELECT (...) FROM X statements. ...
  • 111
0 votes
0 answers
64 views

Unable to upload an excel file to s3 bucket. getting error

My method for uploading a file to s3bucket is as: upload_image_v1: async (files) => { // console.log("files", files); let res_promises = files.map( (file) => new ...
  • 11
0 votes
1 answer
102 views

Backups cannot be restored on AWS RDS - View creation problem. Access denied; you need (at least one of) the SUPER, SET USER privilege(s) for this

The environment is MariaDB 10.6.8, via Amazon RDS. We have all 'admin' rights but not SUPER on AWS databases. I have a view in an application that has it's data stored in a single schema. We use non-...
1 vote
0 answers
58 views

aws rds password change triggers updates to which fields in mysql.user?

I'm using AWS RDS with MySql (5.7.mysql_aurora.2.10.2) Due to internal policy I updated db password via AWS RDS modify DB instance menu. What I can't grasp is, when I updated pw for dev db instance, ...
  • 33
0 votes
0 answers
18 views

Is there a difference in performance loading different datatypes

Context: loading (COPY) a high volume of data to a warehouse (Redshift specifically however curious about all) Question: Is there a performance difference in the different datatypes that I am loading? ...
0 votes
0 answers
108 views

AWS RDS: How do I enable connection for specific IP address?

I've been managing a site for over four years, and initially what I did to allow access to the database for some IP address was just editing an existing Security Group called specificip This is what I ...
  • 101
0 votes
0 answers
48 views

AWS RDS PostgreSQL suddenly consumed all free space

0 I am Migrating table from On-Prem to AWS using AWS DMS tasks. My On-Prem SQL server total DB's sizes are below 1TB But my RDS storage(PostgreSQL) is taking more space 8TB. still it is increasing. I ...
  • 1
0 votes
0 answers
53 views

Can I generate the commands/script to create an RDS Instance from an existing one?

I have a few development and test RDS instances that I need to drop, but would like to be able to recreate them again if so needed (with the same VPC, Options, etc...). Is there a way to, given an ...
  • 133
0 votes
0 answers
304 views

Evaluating AWS RDS Proxy latency overhead

Evaluating some benchmark tests I noticed that there is latency overhead in the transactions to the database using the service provided by AWS: RDS Proxy. The tests were performed against a Postgres ...
  • 101
0 votes
0 answers
14 views

Are AWS Lambda functions actually EC2 instances under the hood?

I’ve heard that AWS security groups are essentially firewalls that can be applied to EC2 instances. Since you can apply security groups to many things in AWS, including Lambdas, I’m wondering if many ...
0 votes
2 answers
135 views

Is it possible to fetch AWS RDS MySQL transaction logs?

I have AWS RDS MySQL instance, without general query logs. Is it possible to get all transaction logs? I understand that MySQL binlog retention time is very short, but obviously, AWS RDS keeps all ...
0 votes
0 answers
168 views

Fastest way to migrate largish (15 TB) PostgreSQL 11 instance to Aurora 14 in AWS

We have an AWS RDS PostgreSQL 11 cluster (currently version 11.13), in which we have a few databases and some 1500 tables. The RDS cluster is of size "db.r5.xlarge", with 4 vCPUs, 32 GB RAM ...
  • 13
0 votes
0 answers
51 views

Migrating AWS RDS SQL Server Express to Azure SQL DB

I have a task to migrate an AWS RDS SQL Server Express instance to an Azure SQL Database. I already assessed the RDS db using Data Migration Assessment Tool and there are not compatibility issues, ...
0 votes
1 answer
69 views

AWS RDS MySQL limit connection

We have multiple databases in a single RDS instance. Because of any reason if we have load in 1 database it stops all the databases. I'm not able to move any of the database because of following ...
  • 115
0 votes
0 answers
27 views

Why is a user randomly created in AWS RDS even though the permissions from any host (%) exist?

I have a MySQL database in RDS. There are separate users which are allowed to connect from anywhere. To be more precise, using MySQL Workbench there are different users listed in "Users and ...
  • 1
0 votes
0 answers
206 views

How to enable replication filters in AWS Aurora 2 (5.7.mysql_aurora.2.10.2)?

I enabled binary log replication in Aurora to replicate from one cluster to another. I went to use the CHANGE REPLICATION FILTER command but get a 1227 error saying I must have the SUPER privilege, ...
1 vote
0 answers
185 views

How to debug a huge MySQL replication lag?

We have an AWS RDS and a corresponding Replica. Once or twice a month the replication lags and does not recover from its own. Our "solution" is to create a new replica and delete the old one....
  • 111
1 vote
1 answer
165 views

dsbulk unload fails on a specific row after running for nearly 2 minutes

We would like to export all data from a Cassandra Keyspace table to CSV with dsbulk. dsbulk unload -k KEYSPACE -t TABLE -f ./application.conf > export_TABLE.csv If we do so, it completes with 1 ...
0 votes
0 answers
125 views

Why does my query take longer when Aurora Serverless is under load?

I have a Aurora Serverless Postgres database (8 to max ACU). I'm running some stress testing against it with about 500 active connections over a period of about 20 minutes. During this time period I'...
  • 678

1
2 3 4 5
10