Take the 2-minute tour ×
Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.

From the jonas slide, I knew that the feature of "Push-Down Join" can make join 70x faster. the most important factor is to eliminate the round trip between API node and data nodes. where all data in the same node, it is not necessary to do inter-node communication. But what if a certain table is partitioned to some group of data nodes, how AQL eliminate the data shipping? can push-down work in sharding scene?

share|improve this question

1 Answer 1

up vote 1 down vote accepted

Pushdown basically means that some part of the job is "pushed down" to data nodes. So performance gain is because:

  • data nodes can filter and process data in much more efficient way than SQL nodes. They "know" more about the physical details of the data
  • we save on SQL node - data node traffic. And latency between those two nodes is relatively high because of the MySQL Cluster design

Inter-data-node latency is much much lower. So data shipping does not have such impact on performance. And all the tables (except some absolute corner cases) are sharded between data nodes. For any data fetch data nodes must find which node holds active part of the required row.

This latency still cannot be ignored. One can find that for the best possible performance Dolphin Interconnect is suggested. But:

  • inter-data-node latency is lower
  • for one data fetch to SQL nodes several communication packets must pass between data nodes. And when filtering is done by data nodes much less packets are needed between data nodes for one processed row.
share|improve this answer
    
that is said it is unavoidable for data shipping between data nodes, but the latency between data nodes can be ignored? –  pythonee Aug 14 '13 at 1:36
    
Expanded answer a bit to answer your comment. –  Mindaugas Riauba Aug 14 '13 at 5:44
    
some document mention that push-down join can make join parallel at data nodes. do you think that is possible? –  pythonee Aug 14 '13 at 8:44
    
Definitely it is possible. Because of pushdown more complex tasks comes from SQL node to data nodes. And data nodes can use different techniques how to complete the task faster. Parallel execution is one of them. –  Mindaugas Riauba Aug 14 '13 at 17:45

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.