6

Polybase is powerful but not really that sophisticated so will not be able to deal with this weird format. You have three options as I see it: Correct the file format at source. Instead of having weird mix of fixed-width and delimited file format, use a standard file format, such as .csv. This format you have where the column delimiter is either carriage ...


4

PolyBase needs TCP enabled to work. By default TCP is disabled in SQL Network Configuration.


3

I'll work from the assumption that all of the tables listed in the query above are external tables pointing to the same external data source. With that in mind, here are a few considerations. Distributed Requests There are two DMVs which will provide a bit more information than what you have currently: sys.dm_exec_distributed_requests and sys....


3

According to a hex editor, your sample file has single line feed (0A) for some line endings and two line feeds as the delimiter between rows: A U-SQL custom extractor might be able to handle this file, but I wonder if we will run into issues with the full 30GB file. Instructions Set up an Azure Data Lake Analytics (ADLA) account if you don't already have ...


2

Creation and maintenance of PolyBase statistics is in the hands of the DBA. The "getting started" guide provides several steps to establish PolyBase, the last of which is CREATE STATISTICS referencing the external table. The documentation on CREATE STATISTICS states: Statistics for external tables When creating external table statistics, SQL Server ...


2

I ended up using sed to cleanse the file zcat myfile.txt.gz | sed -r 's/[ ]+/vin/g'|tr -d '\n'|tr 'vinvin' '\n'|grep -v '^$' > myfile.txt This took care of formatting issues with the source files. Once these files were uploaded to Azure blob storage, the rest was easy. I created external table pointing to files on blob via Polybase and then created ...


1

Polybase does not have this ability. The source file and external table definition must match for it to work. Simply add or remove additional column(s) in your external table definition. Then when you are physically creating your table in the database eg via CTAS, add/remove the additional column(s) from your CTAS statement. Alternately, consider the new ...


1

SQL 2017 is not good with Polybase. Currently, I am using SQL 2019 for this and all is good.


1

MSDN has updated the documentation Option 6: Cloudera 5.1, 5.2, 5.3, 5.4, 5.5, and 5.9 on Linux https://msdn.microsoft.com/en-us/library/mt143174.aspx


Only top voted, non community-wiki answers of a minimum length are eligible