3 Incorporated comment
source | link

For information, I was able to do this using other tools and splitting the workload:

  • To everything work fine, I had to limit each feature in 800 vertices max and 15 km² max. I tried some tools and even a recurring procedure on plsql, but without success. The only thing that I tried and split everything correctly was the Dice feature of ArcMap;
  • I divided my 170k+ rows in chunks of 20 rows and ran six instances of the query in parallel to compute the area using a .net console app;
  • With the area stored, I was able to do some processing, also in chunks and parallel, to compute the value of each single 9kk+ cell.

This process now takes "only" 3 hours to finish.


The ST_Intersects is light. The problem was this calculation:

st_area(st_intersection(grade.the_geom, uso.geom)).

Build the intersection of complex features and calculate the area was the demanding task.

For information, I was able to do this using other tools and splitting the workload:

  • To everything work fine, I had to limit each feature in 800 vertices max and 15 km² max. I tried some tools and even a recurring procedure on plsql, but without success. The only thing that I tried and split everything correctly was the Dice feature of ArcMap;
  • I divided my 170k+ rows in chunks of 20 rows and ran six instances of the query in parallel to compute the area using a .net console app;
  • With the area stored, I was able to do some processing, also in chunks and parallel, to compute the value of each single 9kk+ cell.

This process now takes "only" 3 hours to finish.

For information, I was able to do this using other tools and splitting the workload:

  • To everything work fine, I had to limit each feature in 800 vertices max and 15 km² max. I tried some tools and even a recurring procedure on plsql, but without success. The only thing that I tried and split everything correctly was the Dice feature of ArcMap;
  • I divided my 170k+ rows in chunks of 20 rows and ran six instances of the query in parallel to compute the area using a .net console app;
  • With the area stored, I was able to do some processing, also in chunks and parallel, to compute the value of each single 9kk+ cell.

This process now takes "only" 3 hours to finish.


The ST_Intersects is light. The problem was this calculation:

st_area(st_intersection(grade.the_geom, uso.geom)).

Build the intersection of complex features and calculate the area was the demanding task.

2 deleted 4 characters in body
source | link

For information, I was able to do this using other tools and splitting the workload:

  • To everything work fine, I had to limit each feature in 800 vertices max and 15 km² max. I tried some tools and even a recurring procedure on plsql, but without success. The only thing that I tried and split everything correctly was the Dice feature of ArcMap;
  • I divided my 170k+ rows in chunks of 20 rows and ran the six instances of the query in parallel to compute the area using a .net console app;
  • With the area stored, I was able to do some processing, also in chunks and parallel, to compute the value of each single 9kk+ cell.

This process now takes "only" 3 hours to finish.

For information, I was able to do this using other tools and splitting the workload:

  • To everything work fine, I had to limit each feature in 800 vertices max and 15 km² max. I tried some tools and even a recurring procedure on plsql, but without success. The only thing that I tried and split everything correctly was the Dice feature of ArcMap;
  • I divided my 170k+ rows in chunks of 20 rows and ran the six instances of the query in parallel to compute the area using a .net console app;
  • With the area stored, I was able to do some processing, also in chunks and parallel, to compute the value of each single 9kk+ cell.

This process now takes "only" 3 hours to finish.

For information, I was able to do this using other tools and splitting the workload:

  • To everything work fine, I had to limit each feature in 800 vertices max and 15 km² max. I tried some tools and even a recurring procedure on plsql, but without success. The only thing that I tried and split everything correctly was the Dice feature of ArcMap;
  • I divided my 170k+ rows in chunks of 20 rows and ran six instances of the query in parallel to compute the area using a .net console app;
  • With the area stored, I was able to do some processing, also in chunks and parallel, to compute the value of each single 9kk+ cell.

This process now takes "only" 3 hours to finish.

1
source | link

For information, I was able to do this using other tools and splitting the workload:

  • To everything work fine, I had to limit each feature in 800 vertices max and 15 km² max. I tried some tools and even a recurring procedure on plsql, but without success. The only thing that I tried and split everything correctly was the Dice feature of ArcMap;
  • I divided my 170k+ rows in chunks of 20 rows and ran the six instances of the query in parallel to compute the area using a .net console app;
  • With the area stored, I was able to do some processing, also in chunks and parallel, to compute the value of each single 9kk+ cell.

This process now takes "only" 3 hours to finish.