Questions tagged [amazon-dms]
AWS Database Migration Service (AWS DMS)
32
questions
1
vote
0
answers
113
views
Time zone conversion in DMS from DB2 to Aurora Postgres
Challenge
We are using AWS DMS to convert data from IBM DB2 into AWS Auroa Postgres
IBM DB2 server is set to Eastern time (ET) zone. Timestamp related data inside DB2 is therefore in ET.
Findings
AWS ...
0
votes
0
answers
33
views
Can we use AWS SCT (Schema Conversion Tool) with AWS DMS for ETL and migrating data from MySQL to Amazon Aurora for MySQL?
I want to migrate data from on-premises MySQL/MySQL RDS to amazon Aurora for MySQL. While migrating, I need to convert schema as well as do some ETL to get required schema and data.
In ETL I need to ...
- 21
0
votes
0
answers
197
views
Aws DMS CDC Replication is not working, giving errors
Firstly I successfully migrated DB from mysql mariaDB to PostgreSQL via Aws DMS Task1 with full-load migration type selected.
Now with the Task2 of enabling replication with cdc migration type, its ...
- 101
0
votes
0
answers
159
views
Getting "The provided token has expired" error message in AWS DMS
I have set up a DMS task with the following setup:
RDS MYSQL as source endpoint
and REDSHIFT as target endpoint with "full load and CDC".
setup was working fine for couple of days then ...
- 83
0
votes
0
answers
31
views
Need to create DMS task thru AWS CLI
I have to migrate from MYSQL RDS to REDSHIFT thru DMS. There are 400+ tables in my source MYSQL RDS instance.
i have to create 4 DMS task with 100 tables each thru AWS CLI since thru console it will ...
- 83
0
votes
0
answers
181
views
Getting '.....successive recovery failures' error in DMS while using postgres read replica as source
I'm getting below error in AWS DMS while migrating from Postgres read replica to Redshift. I'm using full load only.The DMS task will run for few mins and some tables will be loaded to target and then ...
- 83
0
votes
0
answers
114
views
Getting ODBC general error in DMS while using postgres read replica as source
I'm getting below error in AWS DMS while migrating from Postgres read replica to Redshift. I'm using full load only.The DMS task will run for few mins and some tables will be loaded to target and then ...
- 83
3
votes
1
answer
257
views
Getting Error 1236 (Could not find first log file name in binary log index file) reading binlog in AWS DMS
I have created two DMS task (full load + CDC) with RDS MYSQL 'Read' replica as source endpoint and REDSHIFT as target endpoint. For both the task source and target endpoint is same. One task with only ...
- 83
1
vote
0
answers
80
views
aws documentation for DMS CDC is confusing
I am following this documentation https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html#CHAP_Source.PostgreSQL.Security to enable CDC using logical replication.
I have pglogical ...
- 61
0
votes
0
answers
169
views
Will SQLNET.EXPIRE_TIME=5 prevent AWS Gateway Load Balancer from timing out after 350 seconds?
We have Amazon Web Services (AWS) Database Migration Service (DMS) tasks erroring out against an on premises Oracle 11.2.0.3 database with ORA-03135 errors. Our network traffic between AWS and the ...
- 221
1
vote
1
answer
821
views
What does MISSING_TARGET means on Amazon DMS (Database Migration Service) validation?
I'm running Amazon DMS from RDS to Aurora, the error when full restarting (never been successful before) from awsdms_validation_failures_v1 tables are:
[
{
"TASK_NAME": "...
- 1,313
0
votes
0
answers
898
views
AWS DMS Task failed with error: Error executing source loop; Stream component failed at subtask 0
I want to migrate my PostgresDB hosted in Citus cloud service to AWS RDS Aurora Postgres. I am using AWS DMS service. Have created task but getting following errors:
Last failure message Last Error ...
- 101
0
votes
1
answer
556
views
How to set PostgreSQL's permission when migrate other database to it on AWS?
We want to migrate an Oracle DB to a PostgreSQL DB in RDS with DMS.
There is an official guide for preparision in the PostgreSQL's DB:
https://docs.aws.amazon.com/dms/latest/sbs/chap-oracle2postgresql....
- 167
0
votes
0
answers
317
views
Continous data replication between two databases(same RDS instance) in AWS RDS
We have multiple databases in a single RDS instance. Forex A, B, C
A and B DB contains the plans table. But some of the fields will be missing under the plans table in B DB.
What is the smart way to ...
- 101
0
votes
0
answers
925
views
AWS DMS migration not migrated Foreign keys, sequences and indexes got renamed
I have used DMS for AURORA PostgreSQL to PostgreSQL SCHEMA Migration. Migrate Data only type is used for DMS activity.
After a Full load of Data, all tables were created on the target endpoint with ...
1
vote
2
answers
1k
views
How does AWS DMS on-going replication works internally?
From the documentation it's mentioned that "DMS collects changes to the database logs using the database engine's native API" and replicates to target. But, I didn't see anywhere that, at ...
- 131
1
vote
0
answers
215
views
How to modify the block size of my S3 parquet files that are being queried in Athena?
I read that adjusting the block size of the parquet files being queried with Athena can affect and possibly improve the performance of the queries. My parquet files for my database are currently ...
- 31.7k
0
votes
1
answer
47
views
Migrate Oracle 12c to 18c in AWS RDS
Currently we have an Amazon RDS Oracle 12c instance and we need to upgrade it to Oracle 18c in a new RDS instance.
What is the correct approach to do this?
Can we set up an 18c instance staright ...
- 147
0
votes
0
answers
764
views
AWS DMS Log growth problem - SQL Server as Source
I have a problem using the AWS Database Migration Service for implementing a transactional replication from SQL Server as a source database engine, a help is highly appreciated.
The 'safeguardPolicy' ...
- 31
3
votes
0
answers
2k
views
Postgresql v10.1 - How to fix ERROR: duplicate key value violates unique constraint
I'm not sure how this happened to this table but have an issue with ERROR: duplicate key value violates unique constraint trying to use AWS's DMS service to move a postgres v10.1 DB to RDS v10
The ...
- 175
0
votes
1
answer
730
views
AWS DMS: 2 million CDCIncomingChanges resolved without target throughput
We're attempting to use AWS DMS to replicate MS-SQL to a customer's on-prem DB, and running into a confusing situation. At 13:00 UTC there are over 2 million CDCIncomingChanges, with CDCTargetLatency ...
- 111
3
votes
1
answer
3k
views
Do not replicate Delete operations AWS DMS
I am using AWS DMS to replicate ongoing changes with SQL Server as a source and target endpoint. The tasks are running and replicating data with low latency. However, I need the tasks configured not ...
- 31
0
votes
1
answer
56
views
Redshift is missing in target database list of AWS SCT Schema Convertion Tool
I want to convert schema of SQL Server database to Amazon Redshift using AWS SCT tool
But as target database, Redshift is not listed
When I use PostgreSQL compatible with IP address of the cluster, ...
- 172
2
votes
0
answers
7k
views
AWS DMS Fails with "Command failed to load data with exit error code 1"
I am trying to Migrate my Postgresql RDS to Aurora Postgresql using DMS.
All my tasks are failing, and I am unable to understand what can be done to help the tasks progress.
The error from cloud ...
- 193
2
votes
1
answer
428
views
AWS DMS - How to select different databases for source and target points for a task
Does aws DMS(Data Migration service) allow to select different databases for source and target points for a task in a replication instance? My source engine is MySQL.
example: I want to move table A ...
- 49
2
votes
1
answer
511
views
Replication from Amazon's RDS to Postgres via DMS errors out
I have a transactional replication from SQL Server 2008 to RDS SQL Server 2016 in 2014 compatibility mode. This works okay. I now want to replicate this RDS database via DMS to Postgresql. This errors ...
- 66
2
votes
2
answers
5k
views
Failure with ongoing replication from SQL Server RDS using AWS DMS
I'm trying to replicate from an SQL Server RDS database using DMS, and running into problems.
I am able to load all data from one table into the destination (~1M rows) as a single 'Migrate existing ...
- 131
3
votes
1
answer
6k
views
AWS DMS endpoint test connection error
I am trying to migrate Microsoft SQL Server databases from on-premise to an RDS instance. I have created a SQL Server 2014 Express Edition instance in RDS and I have SQL Server 2014 Express Edition on ...
- 31
5
votes
1
answer
5k
views
Spike in AWS RDS Oldest Replication Slot lag
Does anyone know about this "Oldest Replication Slot Lag" in AWS RDS? What does it represent?
I have a PostgreSQL instance on RDS (with Logical Replication switched on) for transferring data from ...
- 309
4
votes
1
answer
2k
views
Create event trigger as non-superuser in Postgres
Is it possible to give a role or a set of attributes to a non-superuser in postgres that would allow the user to create an EVENT TRIGGER? Simple example of what (obviously) doesn't work in psql:
=&...
- 141
1
vote
1
answer
385
views
Why did the data type of my DB change?
I had PostgreSQL installation in an EC2 instance. I wanted to migrate to RDS and thus used the DMS service. However, I later came to realize that the data type of the columns have been changed. The ...
- 86
7
votes
2
answers
7k
views
AWS DMS creates tables with no auto_increment
I am migrating my databases to Amazon's RDS using their DMS tool. Pretty much everything looks great except that columns that were marked as auto_increment on the source database did not get marked ...
- 103