0

We have a Firewalled storage account which has a few files that we need to access via Synapse notebook. We do that with below code :

abfss_path = f'abfss://[email protected]/data.csv'        
df = spark.read.load(abfss_path, format='csv',header=True)
display(df.limit(10))

The managed identity and my user (objectid) for the Synapse workspace has ' Storage Blob Data Contributor' role on the storage account.

The below have been enabled:

  • Allow Azure services on the trusted services list to access this storage account.
  • Specify resource instances that will have access to your storage account based on their system-assigned managed identity.

Running the above code works from the pipeline with a synapse notebook task but fails when run from the studio through the notebook cell.

Error : Operation failed: "This request is not authorized to perform this operation.", 403

I have tried everything but can't get it to work.

Via Pipeline

enter image description here

Via Studio Notebook Cell

enter image description here

I can see AAD (pass through) takes precedence when running from studio, so I tried running the session as 'managed Identity'. This doesn't change anything and I keep getting same error.

How to get this working?

Note : I am not using the managed vnet in my synapse workspace.

Improve this question
3
  • Can you chek please if you have the storage acount linked inside synapse workspace? (magane->linked services)
    – MBuschi
    Commented Oct 21, 2022 at 8:10
  • @MBuschi : Yes it is and I can query it via the serverless sql pool with openrowset. The problem is while trying to read the file from sparkpool in a user session. It works from a pipeline via Managed identity.
    – Ramakant Dadhichi
    Commented Oct 21, 2022 at 8:35
  • Can you check if your AAD user and the managed identity have the same role in the storage account IAM role assignament blase?
    – MBuschi
    Commented Oct 21, 2022 at 15:02

1 Answer 1

Reset to default
0

This worked for me, through linked service. You can get more info from here,

%%pyspark
# Set the required configs
source_full_storage_account_name = "teststorage.dfs.core.windows.net"
spark.conf.set(f"spark.storage.synapse.{source_full_storage_account_name}.linkedServiceName", "<lINKED SERVICE NAME>")
sc._jsc.hadoopConfiguration().set(f"fs.azure.account.auth.type.{source_full_storage_account_name}", "SAS")
sc._jsc.hadoopConfiguration().set(f"fs.azure.sas.token.provider.type.{source_full_storage_account_name}", "com.microsoft.azure.synapse.tokenlibrary.LinkedServiceBasedSASProvider")

# Python code
df = spark.read.csv('abfss://<CONTAINER>@<ACCOUNT>.dfs.core.windows.net/<DIRECTORY PATH>')

df.show()
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.