I have been playing around with Apache Spark, firstly I learned PostgreSQL and I have a few queries that I needed to run on Spark. I managed to run them as SQL Strings in Spark SQL, but now I have to perform RDD operations in order to get the same results. I load my data from csv to map. Now I have to select specific columns in those maps, but I do not know how to join them. Second question I have is how to best perform RDD operations in order to get the same results from postgresql queries?
One of the queries:
SELECT Tournaments.TYear,Countries.Name,Max(Matches.MatchDate) - Min(Matches.MatchDate) AS LENGTH FROM Tournaments,Countries,Hosts,Teams,Matches WHERE Tournaments.TYear = Hosts.TYear AND Countries.Cid = Hosts.Cid AND (Teams.Tid = Matches.HomeTid OR Teams.Tid = Matches.VisitTid) AND date_part('year', Matches.MatchDate)::text LIKE (Tournaments.TYear || '%') GROUP BY Tournaments.TYear,Countries.Name ORDER BY LENGTH,Tournaments.TYear ASC