Is it safe to enable Fast DDL in Production

Question

We have a Database running Aurora MySql V5.7. We would like to enable Fast DDL to improve the speed of Adding a Column to a large database. According to the documentation, this feature is Enabled in Aurora Lab mode but isn't recommended for production.

https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Updates.LabMode.html

However I haven't been able to find out why this isn't recommended in production and what risks my application would face by enabling this feature in production.

Can someone provide some additional context for the risks of enabling this feature in production?

Answer 1

Just shooting from the hip, I would not enable it since Aurora has its own universe of options and operations that could cause the garden variety MySQL DBA to break things in their environment.

One thing that comes to mind is a recent post I saw from Percona

Impact of DDL Operations on Aurora MySQL Readers

According to this post, Aurora Readers would be affected when it comes to long running transactions and doing logical backups. If you are not using the Aurora Readers, you still need to proceed with some caution.

Please read that post for more understanding on all the gotchas involved. If doing DDL can break things, doing fast DDL will just break things faster.

Answer 2

I'm obviously speculating, but "lab mode" sounds like a way to enable experimental or "beta¹" features. By definition, such features have not been thoroughly tested and can have unintended side effects and major bugs. Subsequently, all risks of using such features are unknown even to the vendor; you are welcome to help test them, but don't call support if you experience data corruption, cause a core meltdown, or accidentally kill a kitten.

---

1 - Stands for beta than nothin'.

Answer 3

When they say "not recommended for use in production DB clusters," you should take this to mean you should not use that feature in production. That should be clear.

I don't know anything specific about the Fast DDL feature, but in general when a service has that many caveats, it's because development and testing is not complete, and no one knows the potential bugs and risks that are still present in the implementation. They are not withholding knowledge. It's simply that the knowledge does not exist.

I would assume that any or all risks are possible. Their warning is them trying to tell you they can't guarantee any of the following won't happen:

• Fast DDL is not in fact fast in some cases that have not been tested. Which cases these include is unknown.
• Client "locks up" waiting for a blocking DDL operation. In some cases, fast DDL is not possible, and the implementation resorts to non-fast DDL without warning.
• Instance crashes during fast DDL due to code bugs.
• Impact of the DDL affects other unrelated database sessions, due to high resource usage. This could cause poor performance, resource spikes, or crashing.
• DDL performs the wrong action. E.g. you try to change a data type of a column, but it changes to the wrong data type, or other column options are "forgotten" (e.g. NOT NULL or DEFAULT or AUTO_INCREMENT).
• Data loss. Values are changed as a side-effect of the DDL change. Or some rows are deleted. Or the whole table is truncated. Make sure you have a recent backup.

That's just a list of potential scenarios that I can think of. I have not used this feature, and I don't know that any of these do happen with Fast DDL, just that without assurances against it, we should assume they can happen.

No one can tell you the risk level if they don't have enough experience with the feature themselves to know. If the development is still ongoing, then the risk can change at any time.