Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [data-synchronization]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
27 views

Always on availability group removing nodes

We have a SQL 2014 enterprise with 4 node availability group database with 2 Synchronous commit , 1 read only Asynchronous for reporting and 1 DR asynchronous. Due to space shortage we need to remove ...
SQL_NoExpert's user avatar
  • 1,061
0 votes
1 answer
17 views

Pseudo-replication from read-only cloud SQL database to local SQL copy

A vendor has provided us with read-only access to a cloud SQL (2014) database for reporting purposes. We originally built some local reports off this database directly, but that proved too slow and ...
BradC's user avatar
  • 9,984
0 votes
1 answer
64 views

Synchronizing two databases on different servers

I have two databases on different servers. Each one of them contains 350+ tables and users have read and write access on both databases. I want to synchronize them, but in special cases, like if a ...
hanie's user avatar
  • 101
0 votes
0 answers
9 views

Options for automated refresh of 300GB from on-premise to cloud?

We have a requirement to move about 300GB/day of telemetry data from an on-premise collector to a Palantir Foundry Cloud system. Updates should be incremental and occur every 10 minutes. Additional ...
Dr.YSG's user avatar
  • 399
2 votes
1 answer
149 views

Understanding SQL Server Availability Groups: Automatic Seeding of Databases

I'm delving into SQL Server Availability Groups (AGs) and have come across a fascinating feature that seems to simplify the process of setting up replicas – the concept of automatic seeding. From my ...
Just a learner's user avatar
0 votes
2 answers
116 views

How to handle two separate databases? MongoDB and PostgreSQL

We currently have a Postgres database which acts as our core/primary database for our web platform. This database stores registered user accounts (people who have signed up for an account on our site) ...
AccDa's user avatar
  • 11
0 votes
0 answers
154 views

How can I connect SQL Server to MongoDB for Real time data replication?

You will find three tables. Restore those in MSSQL with a db name "source_db". Create a database in Mongodb named "warehouse_db". Now we need to see, for any INSERT/UPDATE/DELETE ...
Vishal Kumar Prajapati's user avatar
3 votes
1 answer
670 views

What are the performance implications of using checksums vs change tracking for SQL Server

Back in the day, an implementation with checksum_agg was made to check if "something changed" in a table in a SQL Server table, to then make updates in another database. The users want to ...
Ola Eldøy's user avatar
0 votes
1 answer
194 views

Can we do incremental sync of MongoDB based on a schedule?

Can we sync a mongodb database to another incrementally based on a schedule? Say, once every night at 12 am? Background: We have a MongoDB replica cluster consisting of 3 instances, all located within ...
esafwan's user avatar
  • 133
0 votes
1 answer
173 views

Sync different table structure in MySQL Database

I have two table in different database host host. activity_table in Host A and presence_table in Host B. presence_table used by binary app where i don't have access to the source code but i have full ...
Abu Dawud's user avatar
0 votes
1 answer
1k views

logical replication initial phase

I am about doing replication for 16 millions record. Just want to ask about the initial phase of logical replication : Do I need to copy the initial table from publisher to subscriber first ? Or is ...
padjee's user avatar
  • 187
1 vote
1 answer
6k views

Monitor logical replication using LSN

How do I monitor a logical replication in Postgresql 12 by seeing the lsn ? What I have done : Inspect some colomns on Publisher and Subscriber. Publisher side : select * from pg_stat_replication; -- ...
padjee's user avatar
  • 187
0 votes
0 answers
617 views

Where, physically, to set the wal_level?

I am setting up replication for a table between 2 Postgresql 12 servers. What I have done : Enable wal_level = logical on the Publisher Create Publisher : '''CREATE PUBLICATION TEST_publication2 FOR ...
padjee's user avatar
  • 187
0 votes
1 answer
96 views

Keeping schemas in sync with a database?

I have been asked to work on a database that has a regretful design and cannot be changed so I would ask that we work within these parameters. This is all in SQL Server 2019. I have SQL Server that ...
Brian Karabinchak's user avatar
0 votes
1 answer
92 views

Is it a bad idea to have a table of deleted UUID's for synchronisation purposes

I have a webapplication and a corresponding mobile application. User can add on the webapp and live DB is auto updated. App can be used offline and add insert and update into local sql lite. Any ...
flyingman's user avatar
3 votes
1 answer
727 views

Best way to synchronize offline data to master when deleting items

I have a application that can work offline as well as the same application via web browser. The web browser is connected to the main DB so always live inserts updates and delete. The application when ...
flyingman's user avatar
-3 votes
2 answers
529 views

Sync databases from different servers [closed]

I have the following setup: One SQL Server on one side, another on the other. The two can't connect, so no possibility to a linked server. It is possible to place the MDFs and LDFs in a shared folder. ...
Florian's user avatar
  • 31
0 votes
2 answers
779 views

Using Cloud and On Premise DB together in an Application

I am developing a textile shop application in .NET Core using the Entity framework. So I am developing it in Azure and using the same application for different shops using Shop ID as the primary key. ...
Sandeep Thomas's user avatar
-1 votes
1 answer
99 views

Query differences between two tables with identical definitions

I have two tables, A and B with identical columns. Table B can contain some rows where data are different from A. How can I write a query that shows rows that should be inserted into B, deleted from B,...
diljaya's user avatar
2 votes
2 answers
298 views

Is partial synchronisation possible between two databases on different servers?

Imagine a scenario where you have a primary instance of a database and a secondary instance that is automatically synchronised - pretty much as with Availability Groups. My question is whether I could ...
youcantryreachingme's user avatar
1 vote
0 answers
112 views

Seeding new data daily in an ever changing database

I've been working with a data team that imports data from suppliers daily into a MySQL DB. The data from the supplier needs to be worked on by them and formatted properly before we can use it in ...
phadaphunk's user avatar
2 votes
2 answers
164 views

How to ship new rows from the source to a target server?

Context We have a SQL Server 'source' where daily 10M rows inserted to a table. No updates. I would like to ship the new rows to a 'target' server, then delete those (sooner or later) on the source ...
g.pickardou's user avatar
1 vote
1 answer
2k views

Why availability group resource goes offline in WSFC?

Sometimes I find the AG Cluster role in offline state which results the AG in resolving state. It doesn't allow the application to access the databases during this time. And then after sometime it ...
Sree Lekha's user avatar
5 votes
1 answer
3k views

Setting up cloud database for syncing to local SQLite database

Background I have a SQL Server database hosted on AWS RDS and there are web applications and WEB APIs that talk to the database. The database is a multi-tenant database and we are currently using SQL ...
keithm's user avatar
  • 151
0 votes
1 answer
44 views

Database syncronization

I am working on a student project and I need some guidance on what solution I can use. The goal of the project is to create a dummy web page that would send dummy data to 2 databases(or read from one ...
Aarlaneth's user avatar
  • 101
0 votes
1 answer
275 views

Database replication from local Postgres to live

I have a software whose data from its local Postgres server has to be synced to live database, so that I can query data from that live Postgres server via api. I am new to Postgres. So after a small ...
Thameem's user avatar
  • 101
1 vote
2 answers
797 views

Is it possible to use txid_current value for sync

I need to expose my data through Rest API for a mobile offline experience. The idea is basically that a mobile client downloads all data to his device. after a week he wants to download only the ...
Michael's user avatar
  • 133
0 votes
1 answer
5k views

What is the max network latency between synchronous-commit AlwaysOn Availability Groups [closed]

Is there a maximum supported network latency (ms) between two synchronous AlwaysOn replica? I need to give this information to network administrators for configuring the link between the two nodes of ...
Gio's user avatar
  • 107
0 votes
1 answer
85 views

Azure Data Sync Between Two Azure Databases

I would like to ask if I use Azure data sync with my two azure databases is there any way that I can still download data tier and install it on my local machine. I am planning to create a sync between ...
Nathan's user avatar
  • 29
0 votes
2 answers
864 views

what are the best ways to sync two databases in sql server or azure sql db?

I have a question like syncing two different databases available on two different servers/instances. lets say Source be the source server and sourcedb is the source database. destinationdb is the ...
Smart003's user avatar
  • 173
0 votes
1 answer
386 views

How to resynchronize replica in Postgresql

I have a server with Centos6 and Postgresql 9.2, I have set up a replica in postgresql to another server of the same characteristics, today in the morning I have an error in the archiver process on ...
miguel ramires's user avatar
1 vote
1 answer
1k views

Will a MERGE between two tables ever be faster than a DROP and SELECT INTO on the second table?

Is there a scenario where it's faster for synchronizing the data from TableA to TableB by using a MERGE query as opposed to dropping TableB and doing a SELECT * INTO TableB FROM TableA?
J.D.'s user avatar
  • 37.9k
1 vote
2 answers
208 views

For AlwaysOn AGs, can a secondary replica cause contention on the primary when the resources of the secondary are maxed out?

Is it possible and in what ways can the secondary replica cause contention against the primary replica when the secondary replica's resources are maxed out (e.g. does the primary "lock" up while ...
J.D.'s user avatar
  • 37.9k
0 votes
1 answer
170 views

How to use a MySQL database both remotely and locally

I cannot find the best solution for my problem. I made a C++ application which runs on a Kiosk under Ubuntu that needs to store and retrieve data from a MySQL database and these same data need to be ...
Marcus Barnet's user avatar
7 votes
1 answer
352 views

Is there a way to determine what kind of data is sitting in the Redo Queue for an AlwaysOn AG Secondary Replica?

Do any of the DMVs or is there another way to expose what kind of data is in the Redo Queue that's in process of being synced up to a secondary replica? (E.g. is it table data and which table, or is ...
J.D.'s user avatar
  • 37.9k
2 votes
1 answer
6k views

What's the cause for an AlwaysOn AG Secondary Replica to have a high Redo Queue Size and Estimated Recovery Time with a good Redo Rate?

SQL14 is the Primary server, SQL16 is the Secondary Replica, and they're setup using synchronous commit availability mode: As of yesterday it appeared that data stopped synchronizing from the primary ...
J.D.'s user avatar
  • 37.9k
1 vote
1 answer
84 views

How would I maintain referential integrity across a database where I use versioned records while keeping data in sync?

I am building a database that consists of a few tables that are used as data sources for a form our application generates. [form] [question] [responses] id id id name ...
Barry Chapman's user avatar
0 votes
2 answers
83 views

Options for syncing from a MASTER table to a COPY table?

I have a MASTER table and a COPY table in separate databases (both are Oracle 12c). I want to sync the MASTER table with the COPY table on a weekly basis. I can think of a few options for syncing ...
User1974's user avatar
  • 1,467
0 votes
1 answer
38 views

Any way of forcing MySQL to use smaller files?

This is actually Maria DB. I'm using it on W10 and Linux Mint 18.3. The issue here is that I recently strated to use Google Backup & Sync to sync between my various machines running W10 (I'm ...
mike rodent's user avatar
0 votes
1 answer
232 views

Best way to replicate MySQL database tables

I'm trying to wrap my head around the many options on how to possibly sync database tables automatically from one database to the next (and next, and next, ...). We're trying to create a very simple ...
frietkot's user avatar
  • 103
1 vote
4 answers
415 views

Integration: Keep two systems in sync

I have a GIS system with 40 tables, ranging from 1,000 to 60,000 rows per table. The tables are the system of record for assets in a municipality. The GIS assets in the tables get integrated to a ...
User1974's user avatar
  • 1,467
1 vote
0 answers
336 views

SQL Server Merge Replication Error: The format of a message during Web synchronization was invalid

I am doing Merge Replication using Web Synchronization. On Subscriber end, there is SQL Express 2012 on Windows 10 Pro. On publisher end, there is SQL Server 2016 on Windows Server 2016. Web ...
Muhammad Bilal's user avatar
0 votes
1 answer
267 views

SQL transaction: Delete between two selects possible?

I have two SELECT statements within a transaction (repeatable read) Select @firstItem =id from myTable Where .... --Do some more magic, so I can't concat the the two queries!!!! and Select * from ...
CSharpProblemGenerator's user avatar
0 votes
2 answers
769 views

Sync database table schema and stored procedures

We have 2 databases. One is at our server, the other is local. We need to sync them both in a manner that local database has all data and server database has no data only empty tables and stored ...
Sadiqabbas Hirani's user avatar
0 votes
0 answers
117 views

Database synchronization 2 times a day -MariaDB

I have 2 different servers. The first one is server1 and the other is server2. On servers, one works for websites (server1) and the other for mobile applications (server2) for security purposes. I ...
mike's user avatar
  • 33
0 votes
1 answer
836 views

Add a new node to Mongo Replicaset using Rsync

I have a 3 shard mongo cluster. Each shard has a 3 node replica set. On each shard, the data size is 3 TB. I want to add 1 more node to each replica set. But its taking lot of time. The initial sync ...
TheDataGuy's user avatar
  • 1,950
0 votes
1 answer
849 views

Change Data Capture in MySQL/PHP by writing to a MySQL table (rather than streaming to Kafka/Rabbit/etc)?

I have a small internal web app I'm developing with PHP and MySQL. I'm looking to sync client webpages to a server by requesting all changes since a certain date. I've found a number of solutions ...
dallin's user avatar
  • 103
0 votes
3 answers
2k views

Sync between two database on azure

I would like to ask a question how to setup azure syncing of two database on azure i need real time data replication to the other database
Nathan's user avatar
  • 29
0 votes
1 answer
2k views

Load data from a temporal table to another temporal table in Azure

Currently we have only a Live Database where all our data resides. I have created a separate Azure server and DB for development. The DEV DB should be some what up to date with the Live DB for ...
Sindu_'s user avatar
  • 103
0 votes
1 answer
296 views

Master Master replication rows not sync properly

I have a wordpress website that run on 3 machines Machine1 | Machine 2 {nginx+lsyncd+mysql} visitor > cloudflare > haproxy ------ ...
iLinux85's user avatar
  • 163