Questions tagged [hadoop]

Apache Hadoop is a framework that allows for the distributed processing of large data sets across clusters of computers. It is designed to scale up from single servers to thousands of machines. The library itself is designed to detect and handle failures at the application layer, so delivering a highly-availabile service on top of a cluster of computers, each of which may be prone to failures.

14 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
1answer
937 views

select two tables and print the columns side by side with matching key in hive

I have the below tables: table_1 name_1 id_1 value_1 avi 1 21 adi 2 X table_2 name_2 id_2 value_2 avi 1 21 adi 2 22 metadata_info: can be read from ...
3
votes
0answers
235 views

Alternatives for storing 365TB of info

I need to upload daily CSV dumps to a blackbox of my choice, and it should be possible to run queries or create filters over the uploaded data. Daily dumps weight around 1TB and are basically id, ...
2
votes
0answers
121 views

How to installation spark standalone mode in ubuntu

I am trying to install spark standalone but show error. How i can solve this problem. Java version :- 1.8.0_131 Spark:- 2.2.0 Hadoop: 2.7.4 bashrc file setting Hadoop file location in local System: /...
2
votes
0answers
141 views

Options for copying data out of continuously updating tables

I work with a pretty large (O(10^1) TB) DB2 (LUW, v. 9.7) database. It's got data coming in on a continuous basis via Golden Gate replication from another DB2 database. It's used for business ...
2
votes
0answers
217 views

Exporting an HBase table which is larger than the available HDFS space?

I have a 1TB hbase table which I need to export. Unfortunately I only have 600GB free in HDFS, so I was hoping to export it to a mounted file share. I've learned that the hbase export can only export ...
2
votes
0answers
343 views

Can we query Accumulo data from Apache Drill

Accumulo uses HDFS where as HBase also works on top of HDFS. We can query HBase data through Drill. So can we query Accumulo data also from Apache Drill ?
1
vote
0answers
71 views

how to access hadoop slaves which have the same IP address but different port numbers

My question is, I'm wondering if it's possible Hadoop can do ssh access with a different port number(not 22) for slaves when it makes a Hadoop cluster. Detail) I tried to make Hadoop cluster with two ...
1
vote
0answers
63 views

mongo-hadoop connector makes duplicate data by numbers of Mongo Sharding

I'm using mongo-hadoop connector which makes that Hadoop can get data from MongoDB and save into MongoDB. I found MongoDB data is duplicated after Hadoop Map-Reduce job with MongoDB data. environment:...
1
vote
0answers
473 views

Apache Phoenix Upsert not working while trying to update null/empty columns

Step 1: Update a column with null value. Upsert into abc.tablename (Rowkeycombo1, Rowkeycombo2, columnname) values (123,to_date('1988-08-06'),null); This works fine and the column then appears to ...
1
vote
0answers
626 views

Enabling JMX For Hadoop HDFS & Also MapReduce

I'm having a hard time figuring out how to enable JMX to submit metrics for HDFS and MapReduce jobs in Hadoop (CDH4). I've seen several links and read through 'The Definitive Guide' and 'Hadoop ...
0
votes
0answers
12 views

Does it make sense that hadoop sqooping would need to sqoop most of the same data all of the time?

I'm a DBA supporting a database that has a large volume of call center data. The team I actually support is the owner of the database, which just imports new data daily, and mostly on this db for ...
0
votes
1answer
301 views

User specific default database in Hive

I am running a Oracle Big Data Appliance platform that has Cloudera EDH 5.9.x running under the hood. My users are mainly using beeswax, which is the Hive query editor app within Hue. When a user ...
0
votes
1answer
217 views

Fastest way to sync (or keep import) 3.5TB data from hadoop to sharded mongodb cluster

There are 3.5TB data in our hadoop cluster(yes on hdfs). And we have newly built a sharded mongodb cluster(the latest 3.x) with 3 mongos, 3 configdb and 3 shards(each shard has 1 primary and 2 ...
-1
votes
1answer
154 views

Suggestions for type of DB to use (Elasticsearch vs MongoDB vs Hadoop)

I am hoping some of you highly skilled DBAs can provide some insight as to the best DB to use for this dataset. This is a relatively simple dataset with no relationship needed between the entries. ...