Questions tagged [aws]
Amazon Web Services.
396
questions
-2
votes
0answers
15 views
Backup to AWS S3
We backup over the weekend our full backups to a backup server with a large EBS volume. Once this is complete we have a second step to invoke a powershell script remotely on the backup server that ...
1
vote
1answer
13 views
What are the risks of upgrading DB major version in replica and then promoting it as master?
Recently AWS decided to force update all MariaDB RDS under 10.3, my RDS is in MariaDB 10.1. We are trying to minimize the downtime and any risk of data out of sync.
Our plan is to create a replica and ...
0
votes
0answers
13 views
restoring sinnge PostgreSQL DB on RDS- AWS
AWS doesn’t make it quite simple to restore just one DB on a RDS. I came up with below plans for DR and would like to know your feedback.
•
Our current RDS-T has D1, D2, D3 and D4 postgresql ...
0
votes
0answers
15 views
What could cause a slow DB connection between AWS and Azure?
Currently we are running with a piece of middleware and our SQL server in AWS. We are looking to move to Azure SQL managed instance. But I am experiencing severely increased insert times when using ...
0
votes
1answer
14 views
How to enable Cloud Watch logs for AWS DMS task?
I am creating AWS DMS task for migrating data from GCP SQL server to AWS Aurora MySQL. How can I attach log group which I created?
I could able to enable cloud watch logs for the task but when I click ...
0
votes
1answer
37 views
How to ensure backups are successfully persisted to S3 when using Ola Hallengren solution on AWS EC2 Instances
Improving Existing Backup Approach By Using Ola Hallengren Backup Solution
Deploying a new backup approach to eliminate dependency on a vendor tool (Cloudberry) that I've leveraged before. In my ...
0
votes
1answer
18 views
My Galera Cluster does not connect across AWS regions
I am building a MariaDB Galera Cluster, on AWS, EC2. Using MariaDB 10.5 and Galera 4.
I have two nodes in the California region, named galera_ca and galera_ca2, and a third node in Oregon name ...
0
votes
0answers
15 views
Design and implement role based access control in DynamoDB
I am learning DynamoDB and trying to implement role-based access control in DynamoDB. Access control is a huge part of the application.
ERD looks something like this:
A little explanation:
Groups ...
0
votes
1answer
19 views
AWS Aurora Mysql different instance types for writer/reader
I have a db.r5.large for the master/writer.
I want to add some readers.
Do they need to be the same instance type as the master? Or can i add for example 3x db.t3.medium ?
0
votes
1answer
43 views
How to setup continuous DB replication from on-prem MS SQL server to PostgreSQL on AWS RDS without reconfiguring source machine?
I am looking for a way to set up continuous data replication between an on-prem Microsoft SQL Server to AWS RDS PostgreSQL. The source table is updated once every 24 hours. The challenge here is that ...
0
votes
0answers
29 views
How to backup a postgresql db to export on AWS lightsail
I want to take a backup from my postgresql db which I use for my Django project.
I'm using ubuntu 18.04 and want to import it in AWS lightsail.
I read pgadmin docs on how to take a backup with a ...
0
votes
0answers
16 views
Unable to connect to AWS RDS Instance
This is the connectivity info:
And this is the configuration:
Public connection is allowed, vpc acl inbound:
What am I missing?
Edit: when I try to connect I get an error as if the endpoint is ...
1
vote
0answers
202 views
DBeaver connection via SSH tunnel
Using DBeaver, I'm trying to connect to my AWS ec2 instance where I've installed a MariaDB database.
I've created a new connection as:
Connection: AWS / Maria DB
Server Host: localhost (because I'm ...
0
votes
2answers
81 views
SQL Server RDS Database Restore from S3 stuck in Restore Mode
Summary:
We are doing a restore of a SQL Server database from S3 on an RDS Instance. And all though the job will say the restore is complete when we go to access the database it's stuck in "...
0
votes
0answers
14 views
Can Redshift process CTE / with chains without making interim tables?
If you are careful about structuring a CTE chain, is Redshift capable of analyzing it and determining that interim results are being used in a linear fashion? If so, it could drop any interim tables ...
0
votes
0answers
20 views
SQL Server HA Design (FCI) - Windows Storage Replication - Test-SRTopology - Physical Memory Error - AWS EC2
I'm trying to set up AlwaysOn Failover Cluster Instance (Not Availability Groups) on 2 VMs(each on different AWS Availability Zone) in AWS-EC2 instances.
To make this happen, I've to set up Windows ...
0
votes
0answers
16 views
Deploying schema changes to AWS RDS hosted Databases
With a database hosted in AWS's RDS, the database is only accessible externally if publicly accessible is set to true and the relevant ingress rules are configured to allow connection from a known ...
2
votes
1answer
177 views
Swap Usage but high Freeable memory
We have an AWS RDS with a MySQL database. Buffer Pool size is only 144GB out of the total available 244GB RAM. However, even at such a low RAM config, we see Swap Usage happening. Does anyone have any ...
0
votes
0answers
17 views
SQL Server Always On in EC2\AWS - Add new replicas how to?
Can someone please answer on this.
we have 2 node WFC with AG group + listener listed to secondary IPs in AWS. one replica in AZ1, another in AZ2.
Now due to perf issues the users added two more nodes....
0
votes
0answers
21 views
Mismatched Identifiers in query for columns with same name but different casing
AWS has a service called Config that allows you to use SQL-like syntax to query your resources. For whatever reason, they changed a field that used to be called dnsname to dNSName when going from v1 ...
-1
votes
1answer
62 views
Problem with mysql/data folder is too big and took the whole place
for the first time I am faced with such a problem that has puzzled everyone.
I have all hosting busy (AWS lightsail) 78GB and after a few analyzes I realized that 90% of the memory is occupied in the ...
0
votes
1answer
28 views
Percona xtradb cluster 5.7 solution on AWS and on premise. Not all nodes are connected to
We have 3 nodes cluster. 2 in virginia and 1 in ohio. The cluster is working fine we do not have any issues. But nothing pop out when we started doing extensive monitoring using datadog. It shows that ...
0
votes
0answers
34 views
SQL Server Availability Groups and CDC
I have a simple SQL Server 2014 Availability Group, one database on a two-node cluster. We need to use CDC to move data to another environment, in my case AWS.
This is a super-critical OLTP very large ...
0
votes
0answers
16 views
Getting (InvalidInputException) when calling the UpdateRelationalDatabaseParameters operation through the awscli
I am trying to increase the value of max_connections of my MySQL DB on my Lightsail account. To do so, I am connecting through ssh using the awscli.
Unfortunately, I receive an error when I type the ...
0
votes
0answers
32 views
Why does my Amazon RDS Postgres server CTE Scan takes much longer than on my local docker postgres server
I just got put on a project to move the local database to AWS RDS. The Queries work great on the local database, but they are extremely slow on amazon RDS.
The execution plan is slightly different as ...
-2
votes
1answer
19 views
What are good options for cloud storage of audit data? (written continuously, rarely queried) [closed]
Cross-post from StackOverflow as I expect it may get closed there because S.O is so nitpicky...
https://stackoverflow.com/questions/63432484/what-are-good-options-for-cloud-storage-of-audit-data-...
0
votes
1answer
230 views
Why are my postgres logical replication workers crashing after syncing a certain amount of data?
I am trying to setup a logical replication between two postgresql 12 DBs run in Docker containers on AWS ECS. I gave the task 2 vCPU and 6 GB of ram so quite luxurious for a DB with ~1e7 rows in two ...
0
votes
0answers
151 views
Does CREATE INDEX on Aurora MySQL 5.6 block INSERT / UPDATE?
I want to add a secondary index to a table on AWS Aurora MySQL engine 5.6.10a in production. The MySQL doc says for an index create like this:
CREATE INDEX name ON table (col_list);
The table remains ...
1
vote
0answers
24 views
Is it bad practice to leave the master database at a lower compatibility level after an in-place upgrade? [duplicate]
Based on this documentation, Microsoft intentionally leaves the master database at the previous compatibility level after an in-place upgrade.
I recently performed an in-place upgrade from 2012 to ...
3
votes
0answers
102 views
RDS MSSQL in-place upgrade how to update master db compatibility level when rdsa account is db owner
Background
My workplace infrastructure team ran a typical in-place upgrade of an RDS MSSQL instance to update the version from 2012 to 2017.
After the upgrade, while logged in as the "master ...
0
votes
0answers
74 views
Password validation in AWS RDS Postgres database
Is there way we can check/set password validation for PostgreSQL RDS database?
like in MySQL we have password validation plugin.
How long password should be/password expiration date etc.
I have ...
0
votes
0answers
7 views
Scalling Down AWS RDS
I have a instance in my AWS RDS running SQL Server, setting up with 16 TB (this is the maximum accepted by AWS), I'd like to change this size to 2TB.
Have anyone some idea how can I do it?
AWS console ...
0
votes
0answers
106 views
Replication external Master to RDS Slave: Connection issue
I am trying to set up replication between my external database (master) to an AWS RDS database (slave). Both are MySQL 5.6.
Please don't mark this as a duplicate. I know others have asked a similar ...
0
votes
1answer
42 views
Snowflake/S3 Pipeline: ETL architecture Questions
I am trying to build a pipeline which is sending data from Snowflake to S3 and then from S3 back into Snowflake (after running it through a production ML model on Sagemaker). I am new to Data ...
0
votes
0answers
54 views
How to modify the block size of my S3 parquet files that are being queried in Athena?
I read that adjusting the block size of the parquet files being queried with Athena can affect and possibly improve the performance of the queries. My parquet files for my database are currently ...
1
vote
1answer
63 views
AWS iops vs disk iops?
Recently devops guy from our company told me that our local mysql stage environment are much faster than aws, because local disk has 10k iops, but our aws rds has only 500 iops.
As far as I ...
1
vote
0answers
17 views
Storing Analytic Data for Multi-Tenet SaaS with AWS Aurora [closed]
I have an app that a user can upload an excel sheet of analytics data to S3. I am wanting to trigger a Lambda function on upload to do some data processing and then write the analytics to a client's ...
-1
votes
1answer
47 views
What value to set for MAX and MIN memory of SQL Server hosted on Bare Metal [duplicate]
Within AWS, Many of our Database servers are being migrated from EC2 Instance to Bare Metal. (Correct me, If I am wrong) In Bare metal, there are two memory limits available for the VM hosted on Bare ...
0
votes
1answer
46 views
AWS RDS MySQL some time get slow performance when update table1 set field='any word' where primarykey=1234
Normally when update table1 set field='any word' where primarykey=1234 take 1sec. or less but sometimes random take much more time possible 500sec. or more.
I was set very well and never see high CPU,...
0
votes
1answer
37 views
Storage problem with MYSQL read replica
I have one master RDS instance of 50 GB storage and created a read replica of the master DB with the same configuration.
I use this read replica only for SELECT operations nothing else. But suddenly ...
0
votes
0answers
64 views
Unable to connect to postgresql EC2 instance from VPN
I have an EC2 instance of timescaledb running on AWS. For security purposes, my team members and I are using OpenVPN so that we have access to it and the same IP address has been defined in the ...
0
votes
1answer
74 views
High write IOPS on MySQL instance
I posted this first in the aws forums before i learned that there's a stackoverflow for DBs but i would ask this here as well.
We recently took over management of a wordpress site using a MySQL DB ...
0
votes
0answers
144 views
MySQL 5.6 - Can I disable SSL for a single user
Server version: 5.6.21-70.1-log
Linux
SSL Keys
ssl-ca=/etc/mysql-ssl/ca-cert.pem
ssl-cert=/etc/mysql-ssl/server-cert.pem
ssl-key=/etc/mysql-ssl/server-key.pem
skip-name-resolve
skip-external-locking
...
0
votes
1answer
159 views
What is the difference between Aurora Mysql 5.7.x and Aurora 2.x.x
I have an RDS instance with Aurora Mysql 5.7.12 and I noticed that there are other versions of Aurora Mysql 5.7, but I don't understand what is the difference between them.
Some of the version ...
0
votes
0answers
5 views
In AWS how to migrate “Microsoft Windows Server 2016” instance to “Microsoft Windows Server 2016 with SQL Server 2017 Standard” AMI
Currently I have "Microsoft Windows Server 2016" instance running in AWS where I have installed evaluation version of SQL server 2017. Now I want to move this server along with SQL database to "...
0
votes
0answers
32 views
AWS RDS Fine Grained Access Control using IAM Authentication
Is it possible to limit a users access to specific 'databases' within an RDS DB cluster using IAM authentication?
Looking at this documentation I can't see any obvious Conditions that would allow ...
0
votes
1answer
184 views
Mongo slow startup after upgrade to 4.2.5
I have upgraded the MongoDB replica set, which consists of 3 members, from 4.0.11 to 4.2.5.
After upgrading, startup lasts about 5 minutes.
Before upgrading it was instant.
It is related to ...
1
vote
1answer
59 views
Speed up AWS PostgreSQL insertion via Python script
I have 120,000 csv inside my AWS EC2 instance, each containing 120,000 rows. I can't insert each of them as is into my AWS RDS postgresql DB, that will be 120,000^2 = 14,400,000,000 records. Each csv ...
1
vote
1answer
2k views
AWS RDS responds with (Got an error reading communication packets). Works locally
I've been on this issue for a couple of days now, I will cover things I've tried at the end. So I'm running a NodeJS server from a GitHub repo using netlify-lamda. I am connecting to an Amazon RDS (...
0
votes
0answers
28 views
MYSQL query working fine in AWS RDS instances with 1 vCPU but hangs if running in instances with 2 vCPU or more
I have this following SQL query below executed in a Lambda function. The Lambda function connected to AWS RDS running MySQL 5.7.22.
When this query is executed against an instance type db.t2.small ...
