From the jonas slide, I knew that the feature of "Push-Down Join" can make join 70x faster. the most important factor is to eliminate the round trip between API node and data nodes. where all data in the same node, it is not necessary to do inter-node communication. But what if a certain table is partitioned to some group of data nodes, how AQL eliminate the data shipping? can push-down work in sharding scene?
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
Pushdown basically means that some part of the job is "pushed down" to data nodes. So performance gain is because:
Inter-data-node latency is much much lower. So data shipping does not have such impact on performance. And all the tables (except some absolute corner cases) are sharded between data nodes. For any data fetch data nodes must find which node holds active part of the required row.
This latency still cannot be ignored. One can find that for the best possible performance Dolphin Interconnect is suggested. But: