Questions tagged [amazon-dms]

AWS Database Migration Service (AWS DMS)

Filter by
Sorted by
Tagged with
0 votes
1 answer
21 views

aurora mysql - migrating large tables

We have write intensive application using aurora mysql 3.02 with 2xlarge instance size in prod. I want to migrate 10 tables (biggest one being 450GB) from 2nd aurora mysql into this prod db. Tried ...
JollyRoger's user avatar
0 votes
0 answers
38 views

MySQL time field type conversion issue with AWS DMS

I have a DB MySQL v5.6 community edition as a source DB and AWS RDS MySQL v5.7 as target in AWS DMS and have an issue with time field type. DMS task is full load and replication. During full load all ...
blagi's user avatar
  • 1
0 votes
0 answers
45 views

Data validation for RDS PostgreSQL migration

I'm planning to replace a bunch of RDS PostgreSQL databases using this method defined by AWS. The DMS task will run will be CDC-only. I came up with data validation requirements for the migration and ...
trouble_bucket's user avatar
1 vote
1 answer
293 views

Time zone conversion in DMS from DB2 to Aurora Postgres

Challenge We are using AWS DMS to convert data from IBM DB2 into AWS Auroa Postgres IBM DB2 server is set to Eastern time (ET) zone. Timestamp related data inside DB2 is therefore in ET. Findings AWS ...
Padraic Renaghan's user avatar
3 votes
1 answer
747 views

Getting Error 1236 (Could not find first log file name in binary log index file) reading binlog in AWS DMS

I have created two DMS task (full load + CDC) with RDS MYSQL 'Read' replica as source endpoint and REDSHIFT as target endpoint. For both the task source and target endpoint is same. One task with only ...
Sumant Kumar's user avatar
1 vote
0 answers
131 views

aws documentation for DMS CDC is confusing

I am following this documentation https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html#CHAP_Source.PostgreSQL.Security to enable CDC using logical replication. I have pglogical ...
Falcon's user avatar
  • 91
0 votes
0 answers
335 views

Will SQLNET.EXPIRE_TIME=5 prevent AWS Gateway Load Balancer from timing out after 350 seconds?

We have Amazon Web Services (AWS) Database Migration Service (DMS) tasks erroring out against an on premises Oracle 11.2.0.3 database with ORA-03135 errors. Our network traffic between AWS and the ...
Bobby Durrett's user avatar
1 vote
1 answer
1k views

What does MISSING_TARGET means on Amazon DMS (Database Migration Service) validation?

I'm running Amazon DMS from RDS to Aurora, the error when full restarting (never been successful before) from awsdms_validation_failures_v1 tables are: [ { "TASK_NAME": "...
Kokizzu's user avatar
  • 1,363
0 votes
0 answers
1k views

AWS DMS Task failed with error: Error executing source loop; Stream component failed at subtask 0

I want to migrate my PostgresDB hosted in Citus cloud service to AWS RDS Aurora Postgres. I am using AWS DMS service. Have created task but getting following errors: Last failure message Last Error ...
Ashish Karpe's user avatar
0 votes
1 answer
732 views

How to set PostgreSQL's permission when migrate other database to it on AWS?

We want to migrate an Oracle DB to a PostgreSQL DB in RDS with DMS. There is an official guide for preparision in the PostgreSQL's DB: https://docs.aws.amazon.com/dms/latest/sbs/chap-oracle2postgresql....
Miantian's user avatar
  • 167
0 votes
0 answers
374 views

Continous data replication between two databases(same RDS instance) in AWS RDS

We have multiple databases in a single RDS instance. Forex A, B, C A and B DB contains the plans table. But some of the fields will be missing under the plans table in B DB. What is the smart way to ...
sdg's user avatar
  • 101
0 votes
0 answers
1k views

AWS DMS migration not migrated Foreign keys, sequences and indexes got renamed

I have used DMS for AURORA PostgreSQL to PostgreSQL SCHEMA Migration. Migrate Data only type is used for DMS activity. After a Full load of Data, all tables were created on the target endpoint with ...
aditya Mehta's user avatar
1 vote
2 answers
2k views

How does AWS DMS on-going replication works internally?

From the documentation it's mentioned that "DMS collects changes to the database logs using the database engine's native API" and replicates to target. But, I didn't see anywhere that, at ...
Pand005's user avatar
  • 151
1 vote
0 answers
264 views

How to modify the block size of my S3 parquet files that are being queried in Athena?

I read that adjusting the block size of the parquet files being queried with Athena can affect and possibly improve the performance of the queries. My parquet files for my database are currently ...
J.D.'s user avatar
  • 37.1k
0 votes
1 answer
54 views

Migrate Oracle 12c to 18c in AWS RDS

Currently we have an Amazon RDS Oracle 12c instance and we need to upgrade it to Oracle 18c in a new RDS instance. What is the correct approach to do this? Can we set up an 18c instance staright ...
thanuja's user avatar
  • 147
0 votes
0 answers
966 views

AWS DMS Log growth problem - SQL Server as Source

I have a problem using the AWS Database Migration Service for implementing a transactional replication from SQL Server as a source database engine, a help is highly appreciated. The 'safeguardPolicy' ...
tOpsDvp's user avatar
  • 31
3 votes
0 answers
2k views

Postgresql v10.1 - How to fix ERROR: duplicate key value violates unique constraint

I'm not sure how this happened to this table but have an issue with ERROR: duplicate key value violates unique constraint trying to use AWS's DMS service to move a postgres v10.1 DB to RDS v10 The ...
Mike's user avatar
  • 175
0 votes
1 answer
912 views

AWS DMS: 2 million CDCIncomingChanges resolved without target throughput

We're attempting to use AWS DMS to replicate MS-SQL to a customer's on-prem DB, and running into a confusing situation. At 13:00 UTC there are over 2 million CDCIncomingChanges, with CDCTargetLatency ...
Nathan's user avatar
  • 111
3 votes
1 answer
4k views

Do not replicate Delete operations AWS DMS

I am using AWS DMS to replicate ongoing changes with SQL Server as a source and target endpoint. The tasks are running and replicating data with low latency. However, I need the tasks configured not ...
tOpsDvp's user avatar
  • 31
0 votes
1 answer
62 views

Redshift is missing in target database list of AWS SCT Schema Convertion Tool

I want to convert schema of SQL Server database to Amazon Redshift using AWS SCT tool But as target database, Redshift is not listed When I use PostgreSQL compatible with IP address of the cluster, ...
Eralper's user avatar
  • 172
2 votes
0 answers
8k views

AWS DMS Fails with "Command failed to load data with exit error code 1"

I am trying to Migrate my Postgresql RDS to Aurora Postgresql using DMS. All my tasks are failing, and I am unable to understand what can be done to help the tasks progress. The error from cloud ...
gvatreya's user avatar
  • 203
2 votes
1 answer
599 views

AWS DMS - How to select different databases for source and target points for a task

Does aws DMS(Data Migration service) allow to select different databases for source and target points for a task in a replication instance? My source engine is MySQL. example: I want to move table A ...
HiteshRathi's user avatar
2 votes
1 answer
606 views

Replication from Amazon's RDS to Postgres via DMS errors out

I have a transactional replication from SQL Server 2008 to RDS SQL Server 2016 in 2014 compatibility mode. This works okay. I now want to replicate this RDS database via DMS to Postgresql. This errors ...
Willem's user avatar
  • 66
3 votes
2 answers
6k views

Failure with ongoing replication from SQL Server RDS using AWS DMS

I'm trying to replicate from an SQL Server RDS database using DMS, and running into problems. I am able to load all data from one table into the destination (~1M rows) as a single 'Migrate existing ...
Greg Sansom's user avatar
3 votes
1 answer
7k views

AWS DMS endpoint test connection error

I am trying to migrate Microsoft SQL Server databases from on-premise to an RDS instance. I have created a SQL Server 2014 Express Edition instance in RDS and I have SQL Server 2014 Express Edition on ...
mohan sahu's user avatar
5 votes
1 answer
6k views

Spike in AWS RDS Oldest Replication Slot lag

Does anyone know about this "Oldest Replication Slot Lag" in AWS RDS? What does it represent? I have a PostgreSQL instance on RDS (with Logical Replication switched on) for transferring data from ...
Zinx's user avatar
  • 309
4 votes
1 answer
2k views

Create event trigger as non-superuser in Postgres

Is it possible to give a role or a set of attributes to a non-superuser in postgres that would allow the user to create an EVENT TRIGGER? Simple example of what (obviously) doesn't work in psql: =&...
matt's user avatar
  • 141
1 vote
1 answer
415 views

Why did the data type of my DB change?

I had PostgreSQL installation in an EC2 instance. I wanted to migrate to RDS and thus used the DMS service. However, I later came to realize that the data type of the columns have been changed. The ...
sameergautam's user avatar
7 votes
2 answers
8k views

AWS DMS creates tables with no auto_increment

I am migrating my databases to Amazon's RDS using their DMS tool. Pretty much everything looks great except that columns that were marked as auto_increment on the source database did not get marked ...
InfernoTK's user avatar
  • 103