Tell me more ×
Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.
up vote 3 down vote favorite
1

I just learned there are "graph databases" out there, which are more suited for two applications "search engine" and especially "gis".

To avoid discussions weather a relational DB is better suited and to make clear what kind of Database is needed, here some more details what I want to use it for:

  1. Application (Search Engine): Handling search engine data in a relational DB is slow because you'll have big tables with enormous indeces. Imagine just a table like 'links' which is just from_pageid INT, to_pageid INT with PRIMARY KEY(from_pageid, to_pageid), INDEX (from_pageid) and INDEX (to_pageid); which would have some 1'000'000 Entries. When using a graph database there are just all "from" and all "to" links for each nodes, with or without index, which are right there without looking up a huge index.
  2. Application (GIS): If you have a map, which consists of nodes (with geographical positions) and links between them (=ways, roads, autoroutes...) you'll have a big "links" table to ask for which links (road, ways,...) can be used at node x. Same problem as for the search engine application.

Ok, now some more specialties:

  1. It should be a free software (at no cost). Reason: I want to use it for research/educational purposes (and I currently have to finance this my own). AND: I possibly want to let it become a commercial product. Therefore a "free for non-commercial use"-solution should be at least effordable, if switching to commercial application...
  2. I would like to use PHP, C/C++ for accessing the DB (preferably both should be possible). If also Java/Android is supported, it would even better.
  3. Operating System: Linux!; I would like a Java based product, as the javaVMs tends to eat up all RAM (only have very limited server resources) and don't free any - even if they could.

What DBs would you suggest to me?

share|improve this question

2 Answers

up vote 3 down vote

While not a Graph or RDBMS based solution, let me suggest a NoSQL database. IMO all of your criteria seem like they could be met with a Cassandra/Solr implementation.

We use Cassandra at work for storing large amounts of data, and we serve it to various applications with a JBoss service layer. Cassandra integrates right in with the Apache Solr search engine. If you are concerned about storing or calculating geographical data Solr offers Spatial Search functions to assist with that.

1- Cassandra and Solr are open source projects with community support. If you choose, you can go the route of commercial support from DataStax. They provide and support DSE (DataStax Enterprise), which is basically an integrated suite of Cassandra, Solr, Hadoop, and several other open source products.

2- We use Hector (Java client for Cassandra) to access Cassandra. I'm pretty sure you can also hit Cassandra from PHP and C++. We've done some R&D work with SolrJ (Java client for Solr). You can also work with Solr via PHP (SolPHP). And last I heard, there is a project in the works for a Solr C++ library (SolC++) under development.

3- Solr and Cassandra both run on Linux. Cassandra is written in Java, so it will run on Windows, too. DataStax supports Solr/Cassandra on both Debian and Red Hat Linux flavors.

share|improve this answer
Hm, Cassandra could be an option. While not semantically fitting perfectly to the problem, Key-Value-Stores are an acceptable alternative. The only trouble I see using Cassandra is that it's a Java server. As explained above: I made very bad experience with memory usage stragies of JavaVMs. They even ignore memory limits set via -Xmx. I was using some Java-software (an open source search engine), which periodically ate up all system memory and crashed the machine -- I was so happy have taken the time to setup a separate vserver for testing it. Anyway... it's an option! Thx! – Stefan K. Aug 17 '12 at 19:13
up vote 2 down vote

Well you could use IBM DB2 LUW (Linux,Unix, Windows) Express-C edition. It is free and has community support. It is the same engine/binaries as DB2 Enterprise Edition, it just has certain features "turned off" and has memory and CPU caps, but for what you are describing, it may suit your needs. If you do find you need more memory/CPU you could always purchase a license from IBM to upgrade it to Express or one of the higher editions.

There is an add on called DB2 Spatial Extender, which has functionality for geo-spatial analysis. According to this chart here, it should be free to download and use, even with Express-C edition. You can download DB2 Spatial Extender here. And here is the Information Center documentation on DB2 Spatial Extender. Use the table of contents to browse.

share|improve this answer
Thanks for your answer, but ... isn't DB2 just a normal relational database which will suffer from performance problems when having lots of entries in a single table (as described in my question)? If it was just the lack of spatial functions I would use postgreSQL + spatial extension; If you still think this would be performant, please explain why... Especially what would be the difference to a mysql solution (which I tried and failed with)? – Stefan K. Aug 17 '12 at 13:39
I'm not sure why you wouldn't think DB2 isn't performant? Like any database, it needs to be tuned properly based on the data it contains. Yes it is a relational database. The spatial extension allows it to work with geospatial information. IBM has other add-ons to allow DB2 to move into other areas like full text search etc. I am not personally familiar with using the spatial extender. I just know it exists and that some places use it. I wouldn't discount DB2 just because it is an RDBMS by default. And the version I pointed out is free. I just offer it as a suggestion. – Chris Aldrich Aug 17 '12 at 14:38
DB2 and other relational databases are fine for the usual applications, where you store "countable" amounts of information into a table. Whereas graph databases are much faster for special applications (as described above). The fundamental difference is data locality. You can put in multiple links (or call it references) from one dataset to other datasets. Now assume a real huge amount of entries. In a relational database those links are stored in a huge table e.g. "links" with 2 IDs (from and to) and lookups take long. For graph databases, the links are stores right inside the entry (=fast). – Stefan K. Aug 17 '12 at 14:52
I guess I'd read on spatial extender to see how it stores the data. I know with DB2 Text Search, it stores the data differently in order to be able to allow full text searches across it. It builds a special set of dictionary/index files that reside outside of the traditional database engine and are only accessed by the DB2 Text Search engine. Spatial Extender may be the same case. It may or it may not store the data relationally. It may store it elsewhere and use a different "engine" to run over it. – Chris Aldrich Aug 17 '12 at 15:16
OK, that is an argument. I'll investigate this in more depth. – Stefan K. Aug 17 '12 at 15:20
show 2 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.