I know how Hadoop works. Is there any possible way to have SQL Server run under Hadoop to build a cluster based on SQL Server and not Drill or Hive.

Just a wild question.

Any good tutorial/documentation?

  • What is the purpose of doing this? It sounds like you have much less complex or better solutions possibly. Also if you need a distributed db in Hadoop HBase or other more optimized services could be used. There's a lot of overhead rewriting the locking/blocking services sql offers though. What's the scope/purpose of the project? – Ali Razeghi Dec 1 '15 at 21:45
  • it is a technical question on the possibilities to utilizes Hadoop to speed up the data process for a database and having a real time fault tolerance, kind of having more advanced RAC in Oracle but for MS SQL, does it make sense? – Data-Base Dec 2 '15 at 6:53
  • no not really at all, unless it's very well planned out and highly optimized, and in that case it would be a big exception. Hadoop is slow and absolutely not designed for real time. There are a few apps written for very specific tasks that could be faster such as TSDB that runs on hbase which then runs on hadoop. Locking and blocking will have to be written in the app by your team. It will do massive bulk processing with tons of volume but it'll need to be highly specialized to do massive throughput near real time. – Ali Razeghi Dec 2 '15 at 16:33
  • That is very interesting... Thanks for clarifying and simplifying it to me :) – Data-Base Dec 3 '15 at 6:16

You cannot run SQL Server under Hadoop but you can query Hadoop data via T-SQL in SQL Server 2016. This next edition of SQL Server includes a technology called PolyBase which allows access of Hadoop data and Azure blob storage from T-SQL. Tutorials are available from Microsoft at the bottom of the linked page.

SSIS for SQL Server 2016 has Hadoop connection managers available also: Hadoop File System Task.

| improve this answer | |
  • sounds good ... so can I migrate an existing data from SQL 2008 R2 to Hadoop through SQL 2016 using PolyBase? – Data-Base Dec 1 '15 at 21:22
  • 1
    If you link the servers you may be able to.... You can definitely export data to hadoop from SQL Server 2016. There are code snippets in the tutorials, just look for the Export Data section: msdn.microsoft.com/en-us/library/mt163689.aspx – Dave Dec 1 '15 at 21:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.