- Error log -- very little activity; don't worry about it.
- redo/undo -- Mandatory; you cannot avoid their use (for InnoDB).
- binlog -- for replication and/or recovery. Clearly state in your benchmark results the setting of log_bin.
- general log -- This should normally be turned off. It is a disk hog. Leave it off for benchmarking.
- slowlog -- It is reasonable to leave it off for benchmarking. (And so-state.) For production systems, I recommend leaving it on. It does not hurt much, even with long_query_time = 1. (That is "seconds".)
- Double-buffer -- Keep on unless the disk subsystem handles it. (Cf "torn page")
- innodb_flush_log_at_commit -- =1 for banking-like security; =2 for more speed, at the expense of very rare loss of data.
The disk subsystem makes a big difference in certain write-intensive apps:
- Plain HDD -- slow
- SSD -- noticeably faster, but non-zero write time.
- RAID with Battery Backed Write Cache -- virtually eliminates I/O time for writes. The BBWC avoids data loss even during power failure.
Benchmarks are a popular way to lie about how great your thing is. What is your goal for the benchmark?
It is somewhat reasonable to compare two hardware setups or versions via comparing benchmarks. However, TPC (etc) benchmarks only test a few focused things. Keep in mind that your app may not be really like any of the TPCs.
Also, any 'real' system has multiple things going on. Benchmarks (like drug trials) mostly punt when it comes to combinations.
Some benchmarks focus on "how many connections can be run before the system keels over?" If you are anywhere near that limit, you are already in trouble, and latency may make the system a failure.