0

Lets start with my question first:

Does having multiple schemas (1 main and 1 daily - having 5-6 tables with same structure as main, but only working for the day and will be synced back with the main one every night) in MySQL (5.7) server can impact its performance ?

More background:

The problem I'm facing with my database is that the application was working fine under certain load (still does but LA goes up to 8), but past few days I'm fighting over CPU usage ... I dig the problem and the reason for that is that there are locks going on and my workers are stuck trying to insert/update new data.

To go even deeper I'm doing at least 1K inserts per minute, up to 2K updates per minute (some are on the previously inserted records), and more than 4-5K selects per minute.

And I'm not talking for simple integer inserts ... lots of strings, timestamps, integers and 6 indexes (total column number in the table is 32)

My server is fairly powerful (huh, fairly).

  • 16GB RAM
  • 1TB SSD
  • 8 Core, Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
  • Software: MySQL 5.7

Part of my my.cnf

innodb_buffer_pool_size = 12G
innodb_buffer_pool_instances = 8
innodb_log_file_size = 30G
innodb_log_buffer_size = 16M
innodb_flush_log_at_trx_commit = 2
innodb_thread_concurrency = 8

and still I'm getting LA of up to 8.00

The problem with all this is because all my workers start every minute and they sort of confront each other.

What I came with is following scenario: - create separate schema with my 'daily' working tables (mostly the SELECTS) and do the inserts/updates in my original schema.

Will that decrease the load on the server?

And one off-topic question: Is LA of up to 8.00 healthy for the server ?

Thanks

P.S. If you need any further info, just let me know I'll share it.

Improve this question

2 Answers 2

Reset to default
0

PROBLEM #1

You innodb_thread_concurrency is too small. SUGGESTION: It should either be 0 or 64. Please read my post from Aug 14, 2017 (MariaDB 10.1.22 to use more RAM rather than CPU) on why.

PROBLEM #2

I don't see innodb_read_io_threads and innodb_write_io_threads mentioned. By default, both are 4. SUGGESTION: I would double both to 8 (Percona Server's default). Please see my post from Jun 20, 2013(MySQL using too much CPU).

PROBLEM #3

Why is your innodb_log_file_size 30G ? InnoDB Redo Logs are written sequentially. IMHO, writing sequential files on an SSD is not that great (See my post from Feb 06, 2014 : MySQL on SSD - what are the disadvantages? and Sep 25, 2014 : What is the best storing device for DB transaction log?). SUGGESTION #1: You could configure InnoDB to have the Redo Logs on a separate disk. SUGGESTION #2: Please make innodb_log_file_size 1G for now. Then, go figure out the proper size by reading my post from Jan 17, 2013 (MySQL 5.5 - Innodb - innodb_log_file_size higher than 4GB combined?) where I point to two posts from Percona's mysqlperformanceblog:

GIVE IT A TRY !!!

As for the one off-topic: Server with 16GB RAM and LA of 8.00 ??? Of course, not !!! Try these suggestions and see if it really helps.

Improve this answer
0

Load Average = 8 is OK, but it usually indicates that you are running out of horsepower. In MySQL, terms, it usually means that you are missing some useful indexes.

"main" and "daily"??? What are you talking about. Perhaps Data Warehouse "Fact" and "summary" tables? If so, then I have lots of advice here.

"Sync back"?? Do you mean you are collecting data in one table (for a day), then copying the data into a bigger table every night? Again, that smells like poor indexes. Or maybe complex queries. Let's see SHOW CREATE TABLE and some of the SELECTs.

17 inserts/sec, 33 updates/sec -- these are not large numbers. Even a spinning HDD can handle such.

6 indexes? Let's see them (SHOW CREATE TABLE). "6" is on the high side; let's see if we can cut back. Note that each index BTree must be updated eventually (cf "change buffer") for each INSERT. Do the UPDATEs change any of the indexed columns? Do you have more than one UNIQUE index? (Note: the PRIMARY KEY counts.) Do you have an AUTO_INCREMENT? Perhaps it could be eliminated.

(I cannot begin to address your question about "multiple schemas" until my questions are answered; I need a better feel for what is going on.)

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.