17

What are some effective ways to perform programmatic performance testing on database operations, especially in environments where the databases themselves do not offer dedicated tools?

For example, in Google App Engine, entire page-loads are evaluated as one operation which may include specific database operations. This problem is also likely present in SQLite and other integrated DBs. As it is difficult to completely abstract the (equivalent of) selects and inserts that need to be tested, are there any recommended database tools to perform more thorough diagnostics on these sorts of queries?

8
  • Do you have direct access to the database in question?
    – Stingervz
    Jan 4, 2011 at 14:51
  • Yes, I'm the one writing the app. And while app performance is a different question, I'm getting some pretty severe performance hits in a nasty query I've written. Jan 4, 2011 at 23:17
  • Sometimes a dozen pairs of eyes is a better diagnostic than a query analyzer ... (ok not that often)
    – jcolebrand
    Jan 7, 2011 at 1:37
  • @Brian I think you would do better on Stackoverflow with this type of question as it's more of a programming question than a DBA one.
    – IamIC
    Jan 8, 2011 at 15:52
  • @IanC Drat. I'm trying to get at the bigtable underlay rather than the performance as a whole. But I will delete if people don't think the question suitable. (I'm also trying to insure that the site isn't oracle/sql-server/mysql all the time) Jan 9, 2011 at 1:11

2 Answers 2

1

It seems to me your problem is that you are trying to test performance metrics that are not well supported in the underlying db. This makes it very difficult to compare performance across systems because the underlying approaches are very different. I don't think it is possible to do apples to apples comparisons just as I don't think you can do an apples to apples comparison of ORDBMS type approaches to RDBMS type approaches. The performance concerns are just too different and if Stonebraker is right that optimizing an ORDBMS for TPC-C tests misses the point, then for systems that are even further apart it is going to be impossible. (I think he is right there, however, only where ORDBMS functionality comes into play.)

I think what you need to be honest is to look at how you would use each system and build a benchmark tool based on the approach you'd take with each. Then you can say at least for that workflow that the benchmark shows something specific. I don't see how you can generalize however. You could further run it in a profiler in order to get additional information about where the time is being spent in the test run on various engines.

However db benchmarking is very difficult to make meaningful in the best of circumstances and when you are comparing dissimilar systems it becomes impossible to generalize.

0

Appstats is the key tool for measuring performance on App Engine. It will show the time used for each RPC, including datastore, memcache, urlfetch and mail requests in a graphical chart. Normally the requests appear as a "staircase" where each request starts at the point the previous request ended, on the next line down.

If you use the advanced asynchronous requests in ndb, you can actually see the requests happen in parallel.

This tool has been an immense help for me in seeing where the time is being spent and how to optimize queries.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.