1

In Azure Databricks I have I have a repo cloned which contains python files, not notebooks.

In Azure Data Factory I want to configure a step to run a Databricks Python file. However when I enter the /Repos/..../myfile.py (which works for Databricks Notebooks) it gives me the error " DBFS URI must starts with 'dbfs:'"

How can I reference a python file from a report which is not in dbfs?

enter image description here

NOTE I see a duplicate question here but the answer was just to wrap it in a Databricks Notebook - OK workaround but when I do it I get "No module named 'my_python_file'"

https://stackoverflow.com/questions/70096408/how-to-create-a-databricks-job-using-a-python-file-outside-of-dbfs

3
  • what databricks runtime is used?
    – Alex Ott
    Dec 1, 2021 at 9:05
  • 8.x but could also setup 9.1 Dec 2, 2021 at 22:12
  • Files in repos are supported only since 8.4, but better to take 9.1
    – Alex Ott
    Dec 3, 2021 at 6:20

1 Answer 1

1

For reference

After all the mucking around, we decided to build a Python package (wheel) which can be imported to Databricks as a library. This can then be installed with %pip install /dbfs/..../*.whl and the package is then accessible to notebooks.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.