Questions tagged [aws]
Amazon Web Services.
406
questions
0
votes
0answers
13 views
Postgres Database flagged as STORAGE FULL whereas there's 45 Gb available
I will start this post by stating that I am not very experienced in database administration, please accept my apologies for any unclear explanation you may find below.
We have a replica postgresql ...
0
votes
0answers
11 views
Timestamp is off by an hour When Migrating from Mysql to Sql server AWS DMS
We are trying to migrate a MySQL database on AWS RDS to SQL Server Database on AWS.
Both the Source and Target Database are created in PST Time zone.
Replication Instance Version : 3.4.4 Mysql Version:...
0
votes
1answer
102 views
How to connect Datagrip with AWS Athena using an assumed role?
I am able to make Athena queries through the AWS CLI. This requires me to assume a role first (I use awsume). The role requires MFA.
Now I want to configure Datagrip for this as well. However there ...
2
votes
1answer
43 views
Backup TO URL S3
Can I do a SQL backup to AWS S3 URL from an EC2 machine
I know that Azure side is possible but I do not know if it is possible AWS side
thank you for your advice
0
votes
0answers
24 views
AWS DMS migration not migrated Foreign keys, sequences and indexes got renamed
I have used DMS for AURORA PostgreSQL to PostgreSQL SCHEMA Migration. Migrate Data only type is used for DMS activity.
After a Full load of Data, all tables were created on the target endpoint with ...
0
votes
0answers
24 views
AWS-RDS database catalog is restricted
I use PostgreSQL 13.2 in AWS-RDS.
The database is up and working properly; I can connect with the user provided by Amazon.
We need to alter the catalog to enable the implicit cast of integer to ...
0
votes
1answer
40 views
delete obsolete archivelog
I have an oracle instance aws recently migrated to RDS aws
I just noticed that my database is set by default in archivelog mode and that the archivelog are not deleted
who has any idea how deleted the ...
0
votes
0answers
24 views
Python / AWS RDS - 300 insert statements taking 10 minutes to complete
I'm using Python 3.7 with SQLAlchemy on Ubuntu 18.04. I have two MySQL Databases - a local instance hosted on bare metal and an Amazon RDS instance. Both are accessible from MySQL Workbench on my ...
0
votes
1answer
17 views
MySQL Replication from On-Premises to AWS
I have an on-premises DB server running MySQL 5.5.37 with multiple databases.
I created database on the AWS RDS.
The idea is to set the replication from on-premises to the AWS, but for only one ...
0
votes
0answers
43 views
Is offsite replication from MySQL AWS RDS with encrypted RDS storage possible? How?
AWS RDS for MySQL now supports encrypted storage backends. This encrypts the MySQL data and binlogs with a key within AWS's key management system, thus causing offsite replication issues because we ...
1
vote
1answer
15 views
Clear PostgreSQL logs in AWS Ubuntu Server
I am trying to clear the logs of an postgresql under /var/log/postgresql/ but I am getting error message as below:
/var/log/postgresql$ sudo > postgresql-11-main.log
-bash: postgresql-11-main.log:...
1
vote
1answer
133 views
subquery must return onl one column in postgresql
I want to get the XML response after the stored procedure call in PostgreSQL. In the below query I was trying to get the output, but it throws an Error: subquery must return only one column. Can ...
0
votes
0answers
20 views
Same query taking different execution times on equivalent RDS instances
The train table has around 60million records all of which have their updated_at timestamp in the year 2021. The schema of the table is as below.
Table "public.train"
...
0
votes
0answers
8 views
Does it make sense to separate records in a document DB (e.g. DynamoDB) for better query performance?
I've recently started working with DynamoDB, and most of my experience is in traditional data warehousing (e.g. sql server). One common pattern which I saw when working with SQL server was that we ...
0
votes
1answer
47 views
how to mongorestore mongoDB 4.2 to documentDB 4.0?
I need to do mongorestore from my mongodb 4.2 to DocumenDb 4.0 compatible i got the command from aws
which is this
mongorestore --ssl --host="abc.aws.com:27017" --username=user --password=...
1
vote
0answers
47 views
SQL Always on Availability Groups Direct Seeding
Setting up a new SQL cluster and HA using SQL Availability Groups.
SQL 2019 STD 2 Node Multi Subnet Basic AG
Primary Node AWS
Secondary Node VMWare on premise
I am using the newer Automatic Seeding ...
2
votes
1answer
30 views
QuasarDB cluster stuck in IOWAIT on AWS EC2 instances
I am experiencing a system freeze in IOWAIT on my QuasarDB cluster.
I am continuously ingesting data at 10 GBit/s using the QuasarDB batch API. After a couple of hours, I have all the qdb threads in ...
0
votes
0answers
26 views
Querying for multiple values when an error message is present in AWS Timestream
I am using AWS Timestream so the DB structure is a bit different than normal but effectively it fairly similar to a normal structure.
What I need to do:
When an error message of a certain type (Ie ...
0
votes
0answers
17 views
AWS RDS Database not in a VPC, no SSL
If an AWS EC2 Server is connecting to an AWS RDS Database, but the database + server are not part of a VPC, and there is no SSL setup on the database...
Does that mean the data is being sent over HTTP ...
0
votes
1answer
18 views
Keeping postgres connection behind load balancer alive during long-running query (vacuum)
Background:
I have a postgres instance behind a load balancer (AWS Classic Load Balancer). I want to access the database from another cluster through the load balancer and run queries. I cannot change ...
1
vote
0answers
24 views
Seeding new data daily in an ever changing database
I've been working with a data team that imports data from suppliers daily into a MySQL DB. The data from the supplier needs to be worked on by them and formatted properly before we can use it in ...
-2
votes
2answers
64 views
Cost of reading and sorting 10GB table at cloud providers [closed]
Say I have a 10 GB table and I need to read it and then sort it by some column. How much does such an operation cost at cloud providers, say AWS or Azure if the pricing is variable? I just need a very ...
-1
votes
1answer
69 views
How to checkdb automatically in AWS RDS?
I'm pretty new to Amazon AWS and RDS. I have migrated my SQL Database to RDS and I have it up and running. I'm now looking to automate maintenance tasks. Anyone with experience with RDS can help me ...
0
votes
0answers
15 views
Can I create another database on AWS read replica?
I am creating an MariaDB instance using AWS RDS I will also be creating a read replica of the DBinstance, what I also need is to host another DB on the replica, is it possible?
0
votes
0answers
12 views
Is an AWS RDS postgres snapshot read consistent - the same as a consistent export in oracle? What happens to transactions in play?
When a snapshot is requested, do all transactions in play get held open (not committed) during the duration of the snapshot creation - only writing the previously committed data to the snapshot?
2
votes
2answers
591 views
MySQL - Can I have quick queries in a table with 200 million rows?
THE PROBLEM
I have following trivial SELECT statement:
select * from `web_notifications` where `user_id` = 123456 order by `created_at` limit 500;
The query is pretty simple (no join statements, no ...
0
votes
1answer
126 views
RDS Postgres chicken-and-egg problem altering roles
I just created an RDS Postgres instance, the user postgres (and it's master password), and connected via psql as user postgres.
Then I ran the script generated from the current system using pg_dumpall ...
1
vote
1answer
18 views
What are the risks of upgrading DB major version in replica and then promoting it as master?
Recently AWS decided to force update all MariaDB RDS under 10.3, my RDS is in MariaDB 10.1. We are trying to minimize the downtime and any risk of data out of sync.
Our plan is to create a replica and ...
0
votes
0answers
21 views
restoring sinnge PostgreSQL DB on RDS- AWS
AWS doesn’t make it quite simple to restore just one DB on a RDS. I came up with below plans for DR and would like to know your feedback.
•
Our current RDS-T has D1, D2, D3 and D4 postgresql ...
0
votes
0answers
59 views
What could cause a slow DB connection between AWS and Azure?
Currently we are running with a piece of middleware and our SQL server in AWS. We are looking to move to Azure SQL managed instance. But I am experiencing severely increased insert times when using ...
0
votes
1answer
28 views
How to enable Cloud Watch logs for AWS DMS task?
I am creating AWS DMS task for migrating data from GCP SQL server to AWS Aurora MySQL. How can I attach log group which I created?
I could able to enable cloud watch logs for the task but when I click ...
0
votes
1answer
150 views
How to ensure backups are successfully persisted to S3 when using Ola Hallengren solution on AWS EC2 Instances
Improving Existing Backup Approach By Using Ola Hallengren Backup Solution
Deploying a new backup approach to eliminate dependency on a vendor tool (Cloudberry) that I've leveraged before. In my ...
0
votes
1answer
82 views
My Galera Cluster does not connect across AWS regions
I am building a MariaDB Galera Cluster, on AWS, EC2. Using MariaDB 10.5 and Galera 4.
I have two nodes in the California region, named galera_ca and galera_ca2, and a third node in Oregon name ...
0
votes
0answers
54 views
Design and implement role based access control in DynamoDB
I am learning DynamoDB and trying to implement role-based access control in DynamoDB. Access control is a huge part of the application.
ERD looks something like this:
A little explanation:
Groups ...
0
votes
1answer
83 views
AWS Aurora Mysql different instance types for writer/reader
I have a db.r5.large for the master/writer.
I want to add some readers.
Do they need to be the same instance type as the master? Or can i add for example 3x db.t3.medium ?
0
votes
1answer
274 views
How to setup continuous DB replication from on-prem MS SQL server to PostgreSQL on AWS RDS without reconfiguring source machine?
I am looking for a way to set up continuous data replication between an on-prem Microsoft SQL Server to AWS RDS PostgreSQL. The source table is updated once every 24 hours. The challenge here is that ...
0
votes
0answers
100 views
How to backup a postgresql db to export on AWS lightsail
I want to take a backup from my postgresql db which I use for my Django project.
I'm using ubuntu 18.04 and want to import it in AWS lightsail.
I read pgadmin docs on how to take a backup with a ...
0
votes
0answers
22 views
Unable to connect to AWS RDS Instance
This is the connectivity info:
And this is the configuration:
Public connection is allowed, vpc acl inbound:
What am I missing?
Edit: when I try to connect I get an error as if the endpoint is ...
1
vote
0answers
995 views
DBeaver connection via SSH tunnel
Using DBeaver, I'm trying to connect to my AWS ec2 instance where I've installed a MariaDB database.
I've created a new connection as:
Connection: AWS / Maria DB
Server Host: localhost (because I'm ...
0
votes
3answers
314 views
SQL Server RDS Database Restore from S3 stuck in Restore Mode
Summary:
We are doing a restore of a SQL Server database from S3 on an RDS Instance. And all though the job will say the restore is complete when we go to access the database it's stuck in "...
0
votes
0answers
23 views
Can Redshift process CTE / with chains without making interim tables?
If you are careful about structuring a CTE chain, is Redshift capable of analyzing it and determining that interim results are being used in a linear fashion? If so, it could drop any interim tables ...
0
votes
0answers
22 views
SQL Server HA Design (FCI) - Windows Storage Replication - Test-SRTopology - Physical Memory Error - AWS EC2
I'm trying to set up AlwaysOn Failover Cluster Instance (Not Availability Groups) on 2 VMs(each on different AWS Availability Zone) in AWS-EC2 instances.
To make this happen, I've to set up Windows ...
0
votes
0answers
72 views
Deploying schema changes to AWS RDS hosted Databases
With a database hosted in AWS's RDS, the database is only accessible externally if publicly accessible is set to true and the relevant ingress rules are configured to allow connection from a known ...
3
votes
1answer
503 views
Swap Usage but high Freeable memory
We have an AWS RDS with a MySQL database. Buffer Pool size is only 144GB out of the total available 244GB RAM. However, even at such a low RAM config, we see Swap Usage happening. Does anyone have any ...
0
votes
0answers
22 views
SQL Server Always On in EC2\AWS - Add new replicas how to?
Can someone please answer on this.
we have 2 node WFC with AG group + listener listed to secondary IPs in AWS. one replica in AZ1, another in AZ2.
Now due to perf issues the users added two more nodes....
0
votes
0answers
24 views
Mismatched Identifiers in query for columns with same name but different casing
AWS has a service called Config that allows you to use SQL-like syntax to query your resources. For whatever reason, they changed a field that used to be called dnsname to dNSName when going from v1 ...
-1
votes
1answer
119 views
Problem with mysql/data folder is too big and took the whole place
for the first time I am faced with such a problem that has puzzled everyone.
I have all hosting busy (AWS lightsail) 78GB and after a few analyzes I realized that 90% of the memory is occupied in the ...
0
votes
1answer
41 views
Percona xtradb cluster 5.7 solution on AWS and on premise. Not all nodes are connected to
We have 3 nodes cluster. 2 in virginia and 1 in ohio. The cluster is working fine we do not have any issues. But nothing pop out when we started doing extensive monitoring using datadog. It shows that ...
0
votes
0answers
43 views
SQL Server Availability Groups and CDC
I have a simple SQL Server 2014 Availability Group, one database on a two-node cluster. We need to use CDC to move data to another environment, in my case AWS.
This is a super-critical OLTP very large ...
0
votes
0answers
26 views
Getting (InvalidInputException) when calling the UpdateRelationalDatabaseParameters operation through the awscli
I am trying to increase the value of max_connections of my MySQL DB on my Lightsail account. To do so, I am connecting through ssh using the awscli.
Unfortunately, I receive an error when I type the ...
