Questions tagged [aws]
Amazon Web Services.
418
questions
1
vote
1answer
20 views
AWS Aurora Postgres 12 Setting Collation
I posted this on stack overflow as well, but didn't get much traction over there. If I am not able to get some thoughts on here, then I will probably open a support ticket with AWS.
I am trying to set ...
0
votes
0answers
19 views
Replacing S3 data lake with Databricks data lake
Very new to the DataBricks technology
We have an AWS S3 data lake.. everything works fine
we work with Big data (new 7GB every month)
I am exploring DataBricks technology
I wonder if we can build our ...
0
votes
1answer
61 views
Moving entire SQL Server 2016 to a new location - best method
At my organization, we have a SQL Server (2016) running on an EC2 instance at Amazon on a Windows with SQL Server license. My boss realized we could save a ton of money by standing up a Windows-only ...
0
votes
0answers
48 views
Oracle Storage Full, How to release space?
i have and AWS RDS Oracle Database wich its storage is full, looks like the cause is SYSAUX tablespace with 10.2GB size
The largest object inside SYSAUX is: WRI$_ADV_OBJECTS (TABLE) checking the table ...
1
vote
1answer
47 views
Intermittent SQLSTATE[HY000] [2003] Can't connect to MySQL server on 'server.here' (110)
I'm suddenly having intermittent 2003 connection errors and I can't find anything that has changed that can explain it. These servers have been running smoothly for years and suddenly two days ago ...
0
votes
0answers
10 views
how to sync one table of AWS RDS to another RDS
I have requirement like need to synchronize one table of data (contain user list) in AWS RDS instance to another RDS Instance.
only particular user's table need to synchronize.
ex: RDS-1 have schema ...
-1
votes
1answer
21 views
How to temporarily disable foreign keys in Amazon RDS Aurora PostgreSQL? [duplicate]
Hi I am using DMS to load data from physical server to AWS Aurora PostgreSQL, but I am getting error for foreign key constraint.
I want to disable using any sp / function. Could you please help me?
0
votes
0answers
10 views
Local SQL server replication to AWS RDS
I have a local licensed SQL server, I need to have an online replica for one of the local databases
I have been reading and checking Amazon Web Service (AWS).
I tried to do SQL Server Replication ...
1
vote
0answers
32 views
Any thoughts on what crashed my database?
A few days ago, our database server, (see specs below), spiked to 4800% CPU usage. The database is about 100GB. For 3 hours we tried to figure out what the issue was. We finally figured out a feature ...
0
votes
2answers
52 views
database changes are lost in the next session
We just move to Amazon RDS. Now all the CREATE, UPDATE, and INSERT commands are reverting in the next session. All the tables are InnoDB and query caching is on. What could be the problem?
0
votes
0answers
28 views
Aurora I/O IOPS vs Billed Volume IOPS
Here is a AWS Aurora PostgreSQL 12 database.
The database is an r5.4xlarge and average load is ~16.
Over 3 hours, the average "Write IOPS" was 40,000 and the average "Read IOPS" ...
0
votes
0answers
55 views
psql connects to AWS RDS instance for co-worker, but not for me
We're on an EC2 VM (the same EC2 VM on the same subnet as the RDS instance) running vanilla PostgreSQL 12.5, and using psql to connect to an AWS RDS PostgreSQL 12.5 instance.
My co-worker and I run ...
0
votes
0answers
10 views
AWS glue - load data by a dynamic condition
I have a concern to apply AWS Glue to my project for below case to see is it possiple on Glue
-I have a AWS RDS Postgresql db and a Resdhift DW
-I want to load data from Postgresql to Redshift but not ...
1
vote
0answers
13 views
Container is running beyond physical memory and killed on request (code 143) [closed]
I ran a query which involved a JOIN and an AVG operation:
SELECT
AVG(user_amount) AS average_user_per_game
FROM
(
SELECT
start_time
,end_time
,COUNT(DISTINCT ID) AS ...
0
votes
0answers
29 views
Remote Nodes is not detecting in cluster validate configuration in sql server 2019
I am not a professional DBA. In my organization, we are trying to implement active active node always on clustering. We have 2 nodes, primary and secondary. After installing WSFC, and during ...
0
votes
1answer
58 views
RDS PostgreSQL -> aws_lambda.invoke error: 'Unable to connect to endpoint'
SELECT * FROM aws_lambda.invoke(
'arn:aws:lambda:us-west-2:123456789012:function:spark-ods-lambda-preprod-myETLTest',
'{"body": "Hello from Postgres!"}'::JSON,
'us-west-2',
...
0
votes
1answer
82 views
Postgres Database flagged as STORAGE FULL whereas there's 45 Gb available
I will start this post by stating that I am not very experienced in database administration, please accept my apologies for any unclear explanation you may find below.
We have a replica postgresql ...
0
votes
0answers
17 views
Timestamp is off by an hour When Migrating from Mysql to Sql server AWS DMS
We are trying to migrate a MySQL database on AWS RDS to SQL Server Database on AWS.
Both the Source and Target Database are created in PST Time zone.
Replication Instance Version : 3.4.4 Mysql Version:...
0
votes
1answer
321 views
How to connect Datagrip with AWS Athena using an assumed role?
I am able to make Athena queries through the AWS CLI. This requires me to assume a role first (I use awsume). The role requires MFA.
Now I want to configure Datagrip for this as well. However there ...
2
votes
1answer
46 views
Backup TO URL S3
Can I do a SQL backup to AWS S3 URL from an EC2 machine
I know that Azure side is possible but I do not know if it is possible AWS side
thank you for your advice
0
votes
0answers
63 views
AWS DMS migration not migrated Foreign keys, sequences and indexes got renamed
I have used DMS for AURORA PostgreSQL to PostgreSQL SCHEMA Migration. Migrate Data only type is used for DMS activity.
After a Full load of Data, all tables were created on the target endpoint with ...
0
votes
0answers
34 views
AWS-RDS database catalog is restricted
I use PostgreSQL 13.2 in AWS-RDS.
The database is up and working properly; I can connect with the user provided by Amazon.
We need to alter the catalog to enable the implicit cast of integer to ...
0
votes
1answer
68 views
delete obsolete archivelog
I have an oracle instance aws recently migrated to RDS aws
I just noticed that my database is set by default in archivelog mode and that the archivelog are not deleted
who has any idea how deleted the ...
0
votes
0answers
35 views
Python / AWS RDS - 300 insert statements taking 10 minutes to complete
I'm using Python 3.7 with SQLAlchemy on Ubuntu 18.04. I have two MySQL Databases - a local instance hosted on bare metal and an Amazon RDS instance. Both are accessible from MySQL Workbench on my ...
0
votes
1answer
20 views
MySQL Replication from On-Premises to AWS
I have an on-premises DB server running MySQL 5.5.37 with multiple databases.
I created database on the AWS RDS.
The idea is to set the replication from on-premises to the AWS, but for only one ...
0
votes
0answers
48 views
Is offsite replication from MySQL AWS RDS with encrypted RDS storage possible? How?
AWS RDS for MySQL now supports encrypted storage backends. This encrypts the MySQL data and binlogs with a key within AWS's key management system, thus causing offsite replication issues because we ...
1
vote
1answer
26 views
Clear PostgreSQL logs in AWS Ubuntu Server
I am trying to clear the logs of an postgresql under /var/log/postgresql/ but I am getting error message as below:
/var/log/postgresql$ sudo > postgresql-11-main.log
-bash: postgresql-11-main.log:...
1
vote
1answer
245 views
subquery must return onl one column in postgresql
I want to get the XML response after the stored procedure call in PostgreSQL. In the below query I was trying to get the output, but it throws an Error: subquery must return only one column. Can ...
0
votes
0answers
21 views
Same query taking different execution times on equivalent RDS instances
The train table has around 60million records all of which have their updated_at timestamp in the year 2021. The schema of the table is as below.
Table "public.train"
...
0
votes
0answers
8 views
Does it make sense to separate records in a document DB (e.g. DynamoDB) for better query performance?
I've recently started working with DynamoDB, and most of my experience is in traditional data warehousing (e.g. sql server). One common pattern which I saw when working with SQL server was that we ...
0
votes
1answer
61 views
how to mongorestore mongoDB 4.2 to documentDB 4.0?
I need to do mongorestore from my mongodb 4.2 to DocumenDb 4.0 compatible i got the command from aws
which is this
mongorestore --ssl --host="abc.aws.com:27017" --username=user --password=...
1
vote
0answers
52 views
SQL Always on Availability Groups Direct Seeding
Setting up a new SQL cluster and HA using SQL Availability Groups.
SQL 2019 STD 2 Node Multi Subnet Basic AG
Primary Node AWS
Secondary Node VMWare on premise
I am using the newer Automatic Seeding ...
2
votes
1answer
39 views
QuasarDB cluster stuck in IOWAIT on AWS EC2 instances
I am experiencing a system freeze in IOWAIT on my QuasarDB cluster.
I am continuously ingesting data at 10 GBit/s using the QuasarDB batch API. After a couple of hours, I have all the qdb threads in ...
0
votes
0answers
41 views
Querying for multiple values when an error message is present in AWS Timestream
I am using AWS Timestream so the DB structure is a bit different than normal but effectively it fairly similar to a normal structure.
What I need to do:
When an error message of a certain type (Ie ...
0
votes
0answers
22 views
AWS RDS Database not in a VPC, no SSL
If an AWS EC2 Server is connecting to an AWS RDS Database, but the database + server are not part of a VPC, and there is no SSL setup on the database...
Does that mean the data is being sent over HTTP ...
0
votes
1answer
22 views
Keeping postgres connection behind load balancer alive during long-running query (vacuum)
Background:
I have a postgres instance behind a load balancer (AWS Classic Load Balancer). I want to access the database from another cluster through the load balancer and run queries. I cannot change ...
1
vote
0answers
25 views
Seeding new data daily in an ever changing database
I've been working with a data team that imports data from suppliers daily into a MySQL DB. The data from the supplier needs to be worked on by them and formatted properly before we can use it in ...
-2
votes
2answers
75 views
Cost of reading and sorting 10GB table at cloud providers [closed]
Say I have a 10 GB table and I need to read it and then sort it by some column. How much does such an operation cost at cloud providers, say AWS or Azure if the pricing is variable? I just need a very ...
-1
votes
1answer
85 views
How to checkdb automatically in AWS RDS?
I'm pretty new to Amazon AWS and RDS. I have migrated my SQL Database to RDS and I have it up and running. I'm now looking to automate maintenance tasks. Anyone with experience with RDS can help me ...
0
votes
0answers
15 views
Can I create another database on AWS read replica?
I am creating an MariaDB instance using AWS RDS I will also be creating a read replica of the DBinstance, what I also need is to host another DB on the replica, is it possible?
0
votes
0answers
12 views
Is an AWS RDS postgres snapshot read consistent - the same as a consistent export in oracle? What happens to transactions in play?
When a snapshot is requested, do all transactions in play get held open (not committed) during the duration of the snapshot creation - only writing the previously committed data to the snapshot?
2
votes
2answers
1k views
MySQL - Can I have quick queries in a table with 200 million rows?
THE PROBLEM
I have following trivial SELECT statement:
select * from `web_notifications` where `user_id` = 123456 order by `created_at` limit 500;
The query is pretty simple (no join statements, no ...
0
votes
1answer
317 views
RDS Postgres chicken-and-egg problem altering roles
I just created an RDS Postgres instance, the user postgres (and it's master password), and connected via psql as user postgres.
Then I ran the script generated from the current system using pg_dumpall ...
1
vote
1answer
20 views
What are the risks of upgrading DB major version in replica and then promoting it as master?
Recently AWS decided to force update all MariaDB RDS under 10.3, my RDS is in MariaDB 10.1. We are trying to minimize the downtime and any risk of data out of sync.
Our plan is to create a replica and ...
0
votes
0answers
22 views
restoring sinnge PostgreSQL DB on RDS- AWS
AWS doesn’t make it quite simple to restore just one DB on a RDS. I came up with below plans for DR and would like to know your feedback.
•
Our current RDS-T has D1, D2, D3 and D4 postgresql ...
0
votes
0answers
79 views
What could cause a slow DB connection between AWS and Azure?
Currently we are running with a piece of middleware and our SQL server in AWS. We are looking to move to Azure SQL managed instance. But I am experiencing severely increased insert times when using ...
0
votes
1answer
35 views
How to enable Cloud Watch logs for AWS DMS task?
I am creating AWS DMS task for migrating data from GCP SQL server to AWS Aurora MySQL. How can I attach log group which I created?
I could able to enable cloud watch logs for the task but when I click ...
0
votes
1answer
181 views
How to ensure backups are successfully persisted to S3 when using Ola Hallengren solution on AWS EC2 Instances
Improving Existing Backup Approach By Using Ola Hallengren Backup Solution
Deploying a new backup approach to eliminate dependency on a vendor tool (Cloudberry) that I've leveraged before. In my ...
0
votes
1answer
133 views
My Galera Cluster does not connect across AWS regions
I am building a MariaDB Galera Cluster, on AWS, EC2. Using MariaDB 10.5 and Galera 4.
I have two nodes in the California region, named galera_ca and galera_ca2, and a third node in Oregon name ...
0
votes
0answers
77 views
Design and implement role based access control in DynamoDB
I am learning DynamoDB and trying to implement role-based access control in DynamoDB. Access control is a huge part of the application.
ERD looks something like this:
A little explanation:
Groups ...
