Apache Hadoop is a framework that allows for the distributed processing of large data sets across clusters of computers. It is designed to scale up from single servers to thousands of machines. The library itself is designed to detect and handle failures at the application layer, so delivering a ...

learn more… | top users | synonyms

1
vote
0answers
7 views

incremental update in HBase using sqoop

i am working on a POC where i need to implement same like SSIS for sql to hadoop. out DW are in SQL Server and we want to create a HBase at central location where all the data from diff DW will be ...
0
votes
0answers
5 views

sqoop dir not found in hortronworks virtualbox

when trying to copy sqljdbc driver to /usr/lib/sqoop/lib it gives error cannot create regular file 'usr/lib/sqoop/lib/' No such File or Directory exist any idea what is missing? downloaded ...
2
votes
0answers
33 views

Options for copying data out of continuously updating tables

I work with a pretty large (O(10^1) TB) DB2 (LUW, v. 9.7) database. It's got data coming in on a continuous basis via Golden Gate replication from another DB2 database. It's used for business ...
0
votes
0answers
26 views

Can we swap out SQL Server with HBase on our .Net application?

In a current application we use SQL Server. The data flow consists of using TVPs for better-than-bulk-insert performance by passing in DataTables. As part of the data addition process, old records ...
0
votes
1answer
21 views

Postgres or MapReduce to Match Reciprocal Needle Pairs from a Stack of Needles?

I have a Postgres (latest version) table with about a million lines (a_lat, a_lon, b_lat, b_lon) that each have about a dozen criteria. These lines go together in reciprocal pairs (A - B, B - A), ...
2
votes
0answers
35 views

Exporting an HBase table which is larger than the available HDFS space?

I have a 1TB hbase table which I need to export. Unfortunately I only have 600GB free in HDFS, so I was hoping to export it to a mounted file share. I've learned that the hbase export can only export ...
0
votes
1answer
59 views

What is the difference between hadoop, mongoDB, Big data? [closed]

What is the difference between Hadoop, MongoDB, Big data ?
1
vote
0answers
73 views

Can we query Accumulo data from Apache Drill

Accumulo uses HDFS where as HBase also works on top of HDFS. We can query HBase data through Drill. So can we query Accumulo data also from Apache Drill ?
1
vote
0answers
111 views

Alternatives for storing 365TB of info

I need to upload daily CSV dumps to a blackbox of my choice, and it should be possible to run queries or create filters over the uploaded data. Daily dumps weight around 1TB and are basically id, ...
0
votes
0answers
24 views

Should I install hadoop on a single node

I just bought a desktop computer with 16GB RAM and 1TB drive, i7-quad core. I will be partitioning off 800GB for Ubuntu so I can do some data science type tasks. Is there any advantage to installing ...
1
vote
1answer
2k views

Small startup: Cassandra + Hadoop or ElasticSearch

To give some context, I am from a small start up (4 team members) and we work in the education space. We've developed an e-learning platform that several schools are already using. We currently have ...
0
votes
1answer
833 views

Problem while connecting to teradata with sqoop

I am trying to connect to Teradat using Sqoop with following command: sqoop import -libjars /usr/lib/sqoop/lib/tdgssconfig.jar,/usr/lib/sqoop/lib/terajdbc4.jar -driver com.teradata.jdbc.TeraDriver ...
0
votes
1answer
508 views

MySQL Rows to Columns to HBase (using Sqoop)

I have a table that looks like this in MySQL: id | date | num | avg I have found that I can use this for rows to columns and it works so far: SELECT CONCAT(id, '-', date), MAX(IF(`num` = ...
0
votes
1answer
112 views

Cassandra with Hadoop

Cassandra uses Cassandra File System (CFS) and Hadoop uses Hadoop Distributed File System (HDFS) I'm already using Pig over HDFS and i would like to install Cassandra. Having additional servers for ...
2
votes
1answer
334 views

How often does the Impala StateStore refresh?

I am using Hive and Impala on the same cluster. I find that when I create new tables, the Impala StateStore does not refresh automatically even after a few hours. I know that I can accomplish this by ...
1
vote
0answers
363 views

Enabling JMX For Hadoop HDFS & Also MapReduce

I'm having a hard time figuring out how to enable JMX to submit metrics for HDFS and MapReduce jobs in Hadoop (CDH4). I've seen several links and read through 'The Definitive Guide' and 'Hadoop ...
0
votes
1answer
274 views

Storing 5TB web server access log vs OLAP DB

We have over 5TB compressed web server log in a HDFS and we often analyse using Hadoop. It is painful to run map reduce on 5TB of data and most importantly, not many developer are familiar with it. ...
1
vote
1answer
251 views

MapReduce performance on a single PC [closed]

I have heard that Hadoop has better performance than MySQL. Until now, I have used relational databases so this is really new technology for me. I have a 6 core PC. Suppose I have a table with 20 ...