I was wondering if there are any good solutions for recording data in a NoSQL database and then converting them over to an RDBMS?

For example, if you wanted to capture some data quickly, like session logs, but then you want to be able to create reports on them later.

My favorite database is Postgres, so if your answer is relevant in Postgres that would be great.

  • At this point postgres has xml, json natively and gis, and hstore (key,value) as extensions. Very little need for a nosql db if using postgres. Jul 18, 2015 at 17:28
  • That said the cqrs pattern is designed to facilitate your question Jul 18, 2015 at 17:29

2 Answers 2


The Quick Answer - Yes. Happens all the time.

There are plenty of good solutions. What solutions are already in your environment? I am helping one client that takes their web site/session activity information from their web application, they write it to xml then deserialize that xml into Hadoop. They then use Hive on top of Hadoop to create aggregations and facts (by joining this log data to reference data from their website that they store in a Postgres database). These aggregtions and facts are then loaded into an RDBMS (they use SQL Server, because of the rich analytics provided in the SQL Server stack).

I would explore Hadoop, look at Hive which is a query engine on top of Hadoop that allows you to write declarative queries that closely resemble SQL and get translated into MapReduce jobs behind the scenes. Then look at Sqoop which is another tool in the Hadoop family that assists you with exporting/importing data.

http://hadoop.apache.org/ http://www.amazon.com/Hadoop-Definitive-Guide-Tom-White/dp/1449389732/ref=sr_1_1?s=books&ie=UTF8&qid=1319160463&sr=1-1 http://www.cloudera.com/blog/2009/06/introducing-sqoop/ http://hive.apache.org/

There are other options out there too...

  • Well that is an extremely complete answer. Doesn't sound too simple, but sounds like it uses a lot of cool technologies.
    – Joe
    Oct 21, 2011 at 15:47
  • Well it isn't "simple" in terms of the number of pieces, etc. but it actually really isn't that complex once you get going. What tool you select for the pieces depends on what you have in your environment and what you are comfortable with. Take a look at those links, check out that book recommendation link.
    – Mike Walsh
    Oct 22, 2011 at 2:20

Not all NoSQL are the same. For instance MongoDB has very good query capabilities. it is very easy to retrieve data, sort data in a manner that reminds you of SQL. So what I am saying is, that you shouldn't be thinking - SQL for reporting, unless you have off-the-shelf tools that require SQL compliant data, of course. Some of the other NoSQL are a bit more involved as far as getting SQL-like results from your store, CouchDB for instance, is not trivial to define a simple where-clause type query. MongodDB - I have used the Python driver - is a cinch. So I am finding it less important to worry about porting my data to a relational store.

  • 3
    Do you have any link or affiliation to MongoDB? Just joined, four answers, all very positive about MongoDB that sound like marketing. You may be correct, but it's poor form if you have an interest...
    – gbn
    Dec 15, 2011 at 12:37
  • Apparently you scared him off there GBN
    – jcolebrand
    Feb 21, 2012 at 16:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.