I just learned there are "graph databases" out there, which are more suited for two applications "search engine" and especially "gis".

To avoid discussions whether a relational DB is better suited and to make clear what kind of Database is needed, here some more details what I want to use it for:

  1. Application (Search Engine): Handling search engine data in a relational DB is slow because you'll have big tables with enormous indeces. Imagine just a table like 'links' which is just from_pageid INT, to_pageid INT with PRIMARY KEY(from_pageid, to_pageid), INDEX (from_pageid) and INDEX (to_pageid); which would have some 1'000'000 Entries. When using a graph database there are just all "from" and all "to" links for each nodes, with or without index, which are right there without looking up a huge index.
  2. Application (GIS): If you have a map, which consists of nodes (with geographical positions) and links between them (=ways, roads, autoroutes...) you'll have a big "links" table to ask for which links (road, ways,...) can be used at node x. Same problem as for the search engine application.

Ok, now some more specialties:

  1. It should be a free software (at no cost). Reason: I want to use it for research/educational purposes (and I currently have to finance this my own). AND: I possibly want to let it become a commercial product. Therefore a "free for non-commercial use"-solution should be at least effordable, if switching to commercial application...
  2. I would like to use PHP, C/C++ for accessing the DB (preferably both should be possible). If also Java/Android is supported, it would even better.
  3. Operating System: Linux!; I would like to avoid a Java based product, as the javaVMs tends to eat up all RAM (only have very limited server resources) and don't free any - even if they could.

What DBs would you suggest to me?

  • 2
    For an association table like this, the common approach is two indexes: PRIMARY KEY(from_pageid, to_pageid) and UNIQUE (to_pageid, from_pageid). 1M rows is not a big table, even for mySQL. If your tried and failed, you did something wrong. But everything depends on what queries/problems you are running/trying to solve. mySQL doesn't have recursive queries, which are very often in graph problems. Jun 29, 2016 at 10:49
  • 2
    And just noticed this is a 4-yeras old question. Which database product did you use in the end and how did it go? Jun 29, 2016 at 10:59

3 Answers 3


Well you could use IBM DB2 LUW (Linux,Unix, Windows) Express-C edition. It is free and has community support. It is the same engine/binaries as DB2 Enterprise Edition, it just has certain features "turned off" and has memory and CPU caps, but for what you are describing, it may suit your needs. If you do find you need more memory/CPU you could always purchase a license from IBM to upgrade it to Express or one of the higher editions.

There is an add on called DB2 Spatial Extender, which has functionality for geo-spatial analysis. According to this chart here, it should be free to download and use, even with Express-C edition. You can download DB2 Spatial Extender here. And here is the Information Center documentation on DB2 Spatial Extender. Use the table of contents to browse.

  • Thanks for your answer, but ... isn't DB2 just a normal relational database which will suffer from performance problems when having lots of entries in a single table (as described in my question)? If it was just the lack of spatial functions I would use postgreSQL + spatial extension; If you still think this would be performant, please explain why... Especially what would be the difference to a mysql solution (which I tried and failed with)?
    – SDwarfs
    Aug 17, 2012 at 13:39
  • 1
    I'm not sure why you wouldn't think DB2 isn't performant? Like any database, it needs to be tuned properly based on the data it contains. Yes it is a relational database. The spatial extension allows it to work with geospatial information. IBM has other add-ons to allow DB2 to move into other areas like full text search etc. I am not personally familiar with using the spatial extender. I just know it exists and that some places use it. I wouldn't discount DB2 just because it is an RDBMS by default. And the version I pointed out is free. I just offer it as a suggestion. Aug 17, 2012 at 14:38
  • DB2 and other relational databases are fine for the usual applications, where you store "countable" amounts of information into a table. Whereas graph databases are much faster for special applications (as described above). The fundamental difference is data locality. You can put in multiple links (or call it references) from one dataset to other datasets. Now assume a real huge amount of entries. In a relational database those links are stored in a huge table e.g. "links" with 2 IDs (from and to) and lookups take long. For graph databases, the links are stores right inside the entry (=fast).
    – SDwarfs
    Aug 17, 2012 at 14:52
  • I guess I'd read on spatial extender to see how it stores the data. I know with DB2 Text Search, it stores the data differently in order to be able to allow full text searches across it. It builds a special set of dictionary/index files that reside outside of the traditional database engine and are only accessed by the DB2 Text Search engine. Spatial Extender may be the same case. It may or it may not store the data relationally. It may store it elsewhere and use a different "engine" to run over it. Aug 17, 2012 at 15:16
  • OK, that is an argument. I'll investigate this in more depth.
    – SDwarfs
    Aug 17, 2012 at 15:20

While not a Graph or RDBMS based solution, let me suggest a NoSQL database. IMO all of your criteria seem like they could be met with a Cassandra/Solr implementation.

We use Cassandra at work for storing large amounts of data, and we serve it to various applications with a JBoss service layer. Cassandra integrates right in with the Apache Solr search engine. If you are concerned about storing or calculating geographical data Solr offers Spatial Search functions to assist with that.

1- Cassandra and Solr are open source projects with community support. If you choose, you can go the route of commercial support from DataStax. They provide and support DSE (DataStax Enterprise), which is basically an integrated suite of Cassandra, Solr, Hadoop, and several other open source products.

2- We use Hector (Java client for Cassandra) to access Cassandra. I'm pretty sure you can also hit Cassandra from PHP and C++. We've done some R&D work with SolrJ (Java client for Solr). You can also work with Solr via PHP (SolPHP). And last I heard, there is a project in the works for a Solr C++ library (SolC++) under development.

3- Solr and Cassandra both run on Linux. Cassandra is written in Java, so it will run on Windows, too. DataStax supports Solr/Cassandra on both Debian and Red Hat Linux flavors.

  • Hm, Cassandra could be an option. While not semantically fitting perfectly to the problem, Key-Value-Stores are an acceptable alternative. The only trouble I see using Cassandra is that it's a Java server. As explained above: I made very bad experience with memory usage stragies of JavaVMs. They even ignore memory limits set via -Xmx. I was using some Java-software (an open source search engine), which periodically ate up all system memory and crashed the machine -- I was so happy have taken the time to setup a separate vserver for testing it. Anyway... it's an option! Thx!
    – SDwarfs
    Aug 17, 2012 at 19:13

Neo4j is a database that sounds a fit for your needs. It is graph based database, there are many drivers for it in many languages and it is built in Java.

In this database, you would specify the relationships within each node, rather than creating tables for joins, like in mySQL.

Not the answer you're looking for? Browse other questions tagged or ask your own question.