Questions tagged [apache-spark]

Use this tag for questions related to the distributed computing framework published by Apache.

Filter by
Sorted by
Tagged with
1
vote
0answers
16 views

Pyspark error inserting into mysql database table that exists

I am trying to insert into an existing mysql table using Pyspark JDBC connection however I get the following error(picture attached) Can I get assistance on this error. The table exist in the MySql ...
0
votes
1answer
12 views

Auto increment field on update

I have a field called: update_count, which I need to reflect the number of times a row has been updated. I have been looking into the SERIAL type for the column which sounds promising, but it has one ...
1
vote
0answers
350 views

HIVE + understanding the hive-metastore logs

we have HDP cluster version - 2.6.4 , and we are runs spark streaming app and we are uses presto cluster in order to run Hive queries when we look on the hivemetastore logs ( under /var/log/hive ) ,...
0
votes
0answers
25 views

Move from SSIS to Azure

I have an ETL solution which we have been doing in conventional SSIS. The job of SSIS is just to create star schema from the application database to dimension and fact tables. There is very little ...
0
votes
1answer
846 views

How to use case and filter with a group by statement and having an aggregate calculation?

I would like to do the following SELECT PO.col1, PO.col2, CASE WHEN PO.col3 <> 8 THEN PO.col4 WHEN PO.col3 = 8 THEN CASE WHEN (ROUND(CAST(...