Questions tagged [databricks]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
-1 votes
1 answer
21 views

How to create "On this day in history" query

I'm using Databricks and I have a table with a list of event from various years. I want to return the event most recent to today's date from each year. For example, Today's date is 6th May and my ...
  • 1
1 vote
1 answer
106 views

DESCRIBE TABLE in databricks piped into dataframe

Does anyone know of a method to pipe the "DESCRIBE TABLE" output in databricks into dataframe? (or other usable format which could be used for further analysis/computation)?
  • 111
1 vote
1 answer
1k views

How to call python file in repo in databricks from data factory outside DBFS?

In Azure Databricks I have I have a repo cloned which contains python files, not notebooks. In Azure Data Factory I want to configure a step to run a Databricks Python file. However when I enter the /...
2 votes
0 answers
414 views

Troubleshoting slow running queries/jobs in Azure Databricks

I have Azure Databricks workspace with cluster configured to run Standard 6.4 runtime (Apache Spark 2.4.5, Scala 2.11). Cluster uses shared metastore (Azure MySQL). I'm trying to figure out possible ...
  • 717