Questions tagged [aws]
Amazon Web Services.
0
votes
0answers
5 views
Integrating MongoDB metrices with Cloud watch
We are hosting mongodb in AWS environment and planning to create incident in Servicenow using AWS Cloud watch alarm facility
Has anyone integrated MongoDB Monitoring metrices with Cloud watch to ...
-3
votes
0answers
18 views
How to get rolling 12 months data in Athena
I have a database like
sno product date sale
1 - 2019-01-16 10
2 - 2019-01-15 20
3 - 2019-01-01 23
4 - 2018-12-29 12
and ...
0
votes
0answers
8 views
Setup mysql innodb group replication in AWS with instance from different region
The plan is to have 4 mysql instances, 3 at US and 1 at Asia as emergency backup.
The problem is, the public ip address (elastic public ip) in EC2 is not part of the system's interface. When the ...
-2
votes
0answers
58 views
Database for building a realtime analytics system for heavy load
I'm starting to think on an implementation for analytics system.
_This system should be able to able to handle about 6,000,000,000 items per 24 hours.
Per each item I need to keep a boolean flag and ...
-1
votes
0answers
18 views
Red indicator current activity issue in MySQL - AWS
I run a MySQL RDS in AWS and I saw that the current activity indicator is reaching the limit, while the RDS in total is not that much loaded (CPU15%, 40GB of SSD are free, user connections are low etc....
1
vote
1answer
20 views
PostgreSQL Materialized View not refreshing
I'm using PostgreSQL in DBeaver and querying data from AWS.
ana.bookings is a materialized view and it shows only the data up to 26th November 2018 even though all the tables and views prior to it ...
1
vote
0answers
24 views
How to backup a local database to cloud database?
I am about to start developing a POS software as my new project with python and electron js. It is going to be a desktop application.
However, my concern is it should work in both cases when desktop ...
1
vote
0answers
27 views
Best Distkey and Sortkey for the tables in Redshift
I have the following tables:
orders:
id => primary key
customer_id =>
store_id =>
order_date => datetime
order_items:
id => primary key
order_id => FK
The query I usually run ...
0
votes
1answer
37 views
Change in bind_ip value is not reflecting
I have mongo installed on Centos instance in AWS and I am connecting to the mongoDB server from my system. To connect from remote system we need to change bind_ip value to 0.0.0.0 in /etc/mongod.conf ...
-1
votes
0answers
14 views
Strongly consistent backup of Mongo with Aurora at the exact same time
At my employer we use AWS Aurora DB & MongoDB Atlas hosted databases together on one application.
So we have nail biting scenarios where records are created on both and typically MongoDB points ...
1
vote
2answers
39 views
How do you know when it's time to upgrade a RDBMS/SQL database infrastructure? What metrics to follow? [closed]
In my specific case, I have just deployed my side project to AWS and using AWS MySQL Tier 2 Micro instance, which is within AWS' Free Tier;
https://aws.amazon.com/ec2/instance-types/t2/
It has: 20 ...
1
vote
1answer
58 views
What database to choose for 1TB of time series? [closed]
I'm saving about 3GB per day of financial orderbook data to a database (currently Mysql). In the coming weeks, I'm going to save trade and quote (bids/asks) data too.
Data is structured, but I'm ...
0
votes
1answer
44 views
What's your data ETL methodology? [closed]
I think this is more like an open question: How do you do the ETL for your company?
We are using MySQL as our database management system and we have a production schema. Every month, when we received ...
0
votes
1answer
23 views
Master slave replication with different mysql version is on both slave and master
Currently, we have MySQL master-slave replication. Both master and slave servers have Mysql 5.5 Currently I need to add one more Master server having version 5.7.
Is it possible to add Multiple ...
1
vote
0answers
16 views
AWS RDS Performance insights - Do DBAs would need a DB user? [closed]
This question is more to clarify or expand my knowledge. Currently, on my work, we're migrating all our systems to AWS. Regarding to databases, we start using performance insights but I'm not really ...
1
vote
1answer
45 views
MongoDB deployment across cloud and On Premises
We are planning to implement MongoDB as a stretch cluster which includes Primary secondary in DC 1, Secondary secondary in DC 2, Arbiter in cloud(DC 3) where DC 1 and DC 2 are on premise servers and ...
0
votes
1answer
169 views
Best way to pg_dump PostgreSQL on RDS to S3
I need to dump my PostgreSQL on RDS to a file on S3 periodically (to use it elsewhere than AWS).
Is there a better way than running a script that runs pg_dump and uploads results to S3 using e.g. ...
-1
votes
0answers
98 views
39001. 00000 - “invalid argument value” error using DBMS_DATAPUMP.add_file
I been trying to use the Oracle Data Pump PL/SQL API to move data to AWS Oracle RDS. My export job is failing on my local installation of oracle database. Local database is 12c and RDS is 11g. I been ...
1
vote
1answer
40 views
Recommended innodb_file_per_table value for AWS MySQL RDS database
We have the following scenario: Our databases are hosted on AWS MySQL 5.6, and we are planning the following migration: we currently have 10 distinct instances with about 50 databases split between ...
1
vote
1answer
98 views
What is the optimal setting for innodb_buffer_pool_instances for a mid level AWS RDS server with 6gb buffer pool?
We have a MySQL 5.7 AWS RDS instance running on a db.m4.large instance. The basic resources for that instance type are:
7.5gb RAM
2 VCPU
RDS defaults to a calculation that allocates 75% of RAM to ...
0
votes
1answer
27 views
Cannot Create or Drop User in MySQL Workbench
OK. I have set up a new MySQL Server in AWS Ubuntu EC2 and migrate databases from the RDS. After importing the tables, stored procedures and functions into the new server, I encountered the attached ...
2
votes
1answer
177 views
Cannot remotely access PostgreSQL on EC2 instance
I was unable to access a remote AWS postgres database via things
like psql -h *EC2ExternalIP* -U *me*. I would get:
psql -h *EC2ExternalIP* -U *me*
psql: could not connect to server: Operation timed ...
1
vote
1answer
21 views
Backup weekly current production db to Historical db server [closed]
I have a production sql db of 600gb and a historical server where i move the full db every month.
Now we want to have only the last 3 months information on the production server, and move every week ...
1
vote
0answers
76 views
AWS RDS PostgreSQL - Managing Permissions
Relatively new to AWS RDS PostgreSQL environment. We are creating a database with multiple schemas. Users are grouped by "schema_role" that have privileges to create objects in assigned schemas.
My ...
0
votes
2answers
104 views
Redshift table level security
I have a common scenario where I need to drop and create an Amazon Redshift table or view (more often with views). There are situations where an ALTER is not going to work, which would preserve ...
2
votes
1answer
265 views
PostgreSQL - can I use logical replication from a replica's WAL?
I would like to use logical replication. However, as PostgreSQL works now, it can eat up the whole disk and then block the database. Especially in Amazon Web Services RDS, this is a problem because ...
1
vote
0answers
45 views
Domain Independent Cluster on AWS
I am trying to work out whether it's possible to create a Domain Independent Workgroup Cluster in AWS (on Windows Server 2016 for use with SQL 2017 Availability Groups). This environment also has no ...
0
votes
1answer
26 views
One SQL Server RDS instance plus one Ubuntu instance cost money on my initial free trial year [closed]
Why do I get charged when I only have one micro instance running as Ubuntu and another one a RDS. Does the extra RDS count as a separate instance? My RDS is Microsoft SQL Server exchange which should ...
2
votes
1answer
105 views
Replication from Amazon's RDS to Postgres via DMS errors out
I have a transactional replication from SQL Server 2008 to RDS SQL Server 2016 in 2014 compatibility mode. This works okay. I now want to replicate this RDS database via DMS to Postgresql. This errors ...
1
vote
1answer
193 views
How to install cstore_fdw inside RDS postgres 9.6
cstore_fdw is foreign data wrapper for columnar storage for postgresql. I want to install it inside RDS postgre 9.6 instance. Is it possible to install inside RDS instance? Thanks
1
vote
1answer
85 views
Can PAGELATCH Waits cause increase in SOS_SCHEDULER_YIELD wait in SQL Server?
We have a high concurrency web server application on which we have some amount of PAGELATCH_UP & PAGELATCH_SH Waits due to tempdb contention. But recently we have noticed the SOS_SCHEDULER_YIELD ...
1
vote
1answer
109 views
I/O errors backing up to an SMB Share from MSSQL Server 2014
I'm having some issues with SQL backups which are being written to an SMB share on a AWS Storage Gateway Appliance from an EC2 SQL Server. I'm consistently getting the same error in the SQL Event log ...
0
votes
0answers
113 views
Wal-e : “ pg_stop_backup still waiting for all required WAL segments to be archived”
I've been trying to use wal-e to take incremental backup of my postgresql database by following this blog. I've a database on my postgresql and using an AWS S3 bucket. while I'm using the following ...
1
vote
1answer
136 views
Are read replicas heavy reading in affect masterDB?
We have a very heavy load in Read side with large DB tables in 2 hours per day that affect to Db so as solutions I have created read replica for it.
If application fetches large heavy data on read ...
0
votes
0answers
19 views
Partially rotating MongoDB collection efficiency
I have a MongoDB collection with approximately 20M objects in it. Nightly, I am deleting (2 + incomplete portion) months of data and replacing them with an import from CSV that uses PyMongo 3.0....
0
votes
0answers
9 views
Amazon RDS for MySQL Backup
Amazon RDS for MySQL states the following:
There is no additional charge for backup storage, up to 100% of your
total database storage for a region. (Based upon our experience as
database ...
1
vote
0answers
50 views
How to connect to a SQL Server from another computer using SSMS with only hostname?
I set up a VM server computer on AWS EC2 and got a SQL Express Server running on it on port 1433. I made sure to allow TCP/IP requests and allowed TCP port 1433 through the firewall along with UDP ...
1
vote
2answers
255 views
Significantly Slower DB Dumps on AWS RDS After Turning on Encryption?
Turning on encryption on an AWS RDS MySQL instance results in significantly slower database dumps (using mysqldump).
mysqldump -u test -p -h xxx.aws.com > dump.sql
Is there a reason for ...
0
votes
0answers
14 views
Database for storing frequency of access token usage
I would like to create a database to store how often access tokens in my system are being used.
I was thinking of a data model like so -
Access Token
from (basically I will store this hourly, so ...
0
votes
1answer
184 views
Mongodb Cross-region failover replication aws
I have deployed 3 MongoDB nodes(node-1, node-2, node-3) in AWS. The first two nodes (node-1, node-2) are deployed in Virginia region and the third node(node-3) is deployed in Oregon region and enabled ...
1
vote
1answer
59 views
AWS VS Azure , which one is preferable for SQL Server 2012 standard version? [closed]
I want few recommendations while considering azure vs aws for our current database which is SQL Server 2012 standard edition.
Currently we are using AWS RDS for managing our database, recently we had ...
1
vote
0answers
563 views
Queries running very slowly in Amazon RDS PostgreSQL read-replica
We have been maintaining a project which has both web and mobile application platform. The backend of the project is developed in Django 1.10 and deployed in AWS.
At the beginning, when there were ...
-1
votes
1answer
25 views
How to backup documents to s3 in a particular collection older than specific dates in a mongodb collection where timeseries data are stored? [closed]
I have mongodb database.
I have a collection in that stores time series data, time values are stored as ISODate.
Older than few days are not required on database, but required to be backed up in aws ...
2
votes
1answer
120 views
Postgres “session variables” kludge on AWS RDS
I am trying to migrate a PostgreSQL database to AWS RDS.
The problem I ran into was that the database in question makes extensive use of the SET kludge.variable TO 'foo' hack, that is, it uses the ...
0
votes
1answer
28 views
Unable to see measurements in influxdb
I am new to influxdb and I am having this trouble to show measurements.
Our setup is that prometheus(aws ec2) is writing data to Influxdb(aws ec2). When I execute 'SHOW MEASUREMENT' I cannot see ...
1
vote
1answer
84 views
MongoDB write lock every n seconds during data import
I am experiencing an issue with a data import in a sharded cluster and I'd like some help to investigate and maybe find where the bottleneck is in my cluster.
So I'm hosting a sharded cluster (3 ...
1
vote
0answers
33 views
Mongodb secondary going down
I have a problem with my secondary, which is constantly going down, in normal replica set (primary, secondary, arbiter).
What I saw while investigating that there is one long lasting operation in ...
1
vote
1answer
35 views
Guideline for MySQL free space
What is the guideline for how much free space to keep available on a MySQL AWS RDS instance (SSD)? In the past for an on-premises SQL Server install we used to use something like 2x the largest table ...
1
vote
1answer
72 views
After adding indexes to an AWS RDS MySQL DB getting huge/crippling spikes in IOPS
So after looking at the production database, I realised that several queries were running utilizing no indexes at all, and as such, were row-scanning close to 4 million rows. :/
I added the required ...
0
votes
1answer
208 views
Unable to conditionally commit/rollback transaction
I am trying to create a reuable SQL script that I can run that will guard against unintentional db changes based on the expected number of affected rows (not looking to create a sproc/func at this ...
