Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,652 questions
0 answers
Unable to connect serverless sql from Arcgis Pro client (SQL Server Authentication)
It was stated in https://learn.microsoft.com/en-us/azure/synapse-analytics/sql/get-started-ssms, SSMS is partially supported starting from version 18.5, you can use it to connect and query only. I able to use SSMS to connect to serverless sql. But,…
asked 2024-07-11T10:39:00.0366667+00:00
Yap Chun Hing
5
Reputation points
commented 2024-07-11T13:30:58.73+00:00
Harishga
5,825
Reputation points Microsoft Vendor
0 answers
Getting connection issue while connecting to Synapse Dedicated pool from Notebook using Linked Service
I am using the Linked service with spn and certificate and test connection is successful. But when I use the same linked service in Synapse Notebook getting below error. linked service is created for Synapse Analytics and same is working if I change to…
asked 2024-07-08T05:00:54.1133333+00:00
Venkateswarlu Pottapalli
0
Reputation points
commented 2024-07-11T07:57:50.3833333+00:00
PRADEEPCHEEKATLA-MSFT
84,936
Reputation points Microsoft Employee
0 answers
Custom logging in Azure Synapse Datapipeline to to log analytics Workspace - Help Requested
Requirement:- Want to add custom logging in azure synapse pyspark job using python logging module and spark config Getting error while implementing the same in the pyspark job ModuleNotFoundError: No module named 'pyspark.SparkConf pyspark script for…
asked 2024-06-26T23:18:27.1633333+00:00
Johny Yagappan
20
Reputation points
commented 2024-07-11T07:25:52.6166667+00:00
PRADEEPCHEEKATLA-MSFT
84,936
Reputation points Microsoft Employee
0 answers
Synapse custom event trigger publish is not happening on event grid with master branch
Hi all, I have requirement to trigger synapse pipeline when I publish event on event grid. Without git integration when I configure the Trigger and click on publish button, subscription is appearing on Event grid and trigger is working fine. With git…
asked 2024-07-04T07:51:20.4733333+00:00
Madhu Challapalli
20
Reputation points
edited a comment 2024-07-11T05:51:59.29+00:00
phemanth
8,480
Reputation points Microsoft Vendor
0 answers
Sparkpool library Error for adding a zen-engine 0.22.0 in requirement.txt
Hi, PySpark version: 3.3.1.5.2.20240522.3 Python version: 3.10.6 | packaged by conda-forge | (main, Aug 22 2022, 20:35:26) [GCC 10.4.0] I am getting an error when adding zen-engine in requirements.txt in sparkpool in azure synapse spark pool. The…
0 answers
synapse dataframe writing to oracle with JDBC never quits though it has finished writing. It results into a pipeline failure. sometimes similar code works fine.
Py4JJavaError: An error occurred while calling o4089.save. : java.sql.SQLRecoverableException: IO Error: Connection reset by peer at oracle.jdbc.driver.T4CConnection.logoff(T4CConnection.java:1050) at…
asked 2024-07-09T15:26:35.2066667+00:00
Saini, Vijyant
0
Reputation points
commented 2024-07-10T23:29:03.6466667+00:00
BhargavaGunnam-MSFT
28,931
Reputation points Microsoft Employee
0 answers
How to read data from blob storage using Linked Service token in Synapse
Hello, I am trying to use Linked Service token to read from an azure blob storage (NOT AZURE DATA LAKE STORAGE GEN2) in a Synapse Notebook, I created a linked Service and test the connection and it was succesfull but when I am trying to read using spark…
0 answers
synapse notebook
Hi , how we can add link services created in synapse in notebook available azure synapse link servicename :test2 select * from test1.table1
0 answers
After some time, viewing pipeline log links lead to wrong page
This is something I consistently experience if I use the same Synapse Analytics Workspace for a decent amount of time: My workflow is: Monitor Pipeline Runs Click on a run of interest This shows me the pipeline that ran and all the individual steps…
asked 2024-06-27T14:13:06.36+00:00
Andrew Chung
40
Reputation points Microsoft Employee
commented 2024-07-04T06:34:42.9133333+00:00
phemanth
8,480
Reputation points Microsoft Vendor
0 answers
azure synapse
hey i try to get sdk call that return me the info about the Azure Active Directory i cant find the sdk call for this
asked 2024-06-26T08:13:52.66+00:00
Tommy Mordehay
0
Reputation points
commented 2024-07-03T08:29:53.3366667+00:00
PRADEEPCHEEKATLA-MSFT
84,936
Reputation points Microsoft Employee
0 answers
how to generate token in synapse notebook using identity
how to generate token in synapse notebook using identity. Why below code is not working? import requests def get_token_for_risk_api(): resource = "https://xxxxxx" user_managed_client_id = "xxxx" api_version =…
asked 2024-06-24T07:59:14.77+00:00
Nitin Khedekar (WIPRO LIMITED)
0
Reputation points Microsoft Vendor
commented 2024-07-01T13:30:19.0266667+00:00
Smaran Thoomu
12,355
Reputation points Microsoft Vendor
0 answers
Azure Synapse Link for dataverse to bring Dynamics 365 data to ADLS and Synapse for analytical purpose - Facing Issues and inconsistencies in the feature
We are using Synapse Link for Dataverse feature to bring dynamics 365 data to ADLS and Synapse Analytics. We have below open issues due to which we are unable to finalize the solution: In F&O linked dataverse environment, we have created a synapse…
0 answers
Mssparkutils.credentials.getSecretWithLS to get Certificate and create CertificateCredential in Synapse
I need to create a CertificateCredential from azure.identity, I have a PKCS12 certificate saved in a Key Vault and I am reading it using mssparkutils with LinkedService, official documentation mentioned you can create a CertificateCredential with the…
0 answers
Unable to list the directory when create view for delta
Assuming, I have the a single spark dataframe outputted as DELTA into two folders using pyspark: folder A and folder B in the same blob storage Gen2 For Folder A and Folder B, both have external data source created with same credentials using sql…
asked 2024-06-18T10:04:42.67+00:00
Yap Chun Hing
5
Reputation points
commented 2024-06-19T19:38:43.4266667+00:00
BhargavaGunnam-MSFT
28,931
Reputation points Microsoft Employee
0 answers
Why has Synapse Analytics pipeline reverted to a previous version of a notebook?
I made some changes to a couple of Synapse Analytics notebooks on Friday. These notebooks are part of an existing pipeline. I committed the changes, PRd and merged the branch as normal etc. The pipeline runs daily. It ran perfectly fine on Saturday,…
asked 2024-06-17T11:14:17.5533333+00:00
Daniella Quinn
0
Reputation points
commented 2024-06-18T09:04:57.4233333+00:00
Harishga
5,825
Reputation points Microsoft Vendor
0 answers
Avoiding metric inflation during data processing in Dedicated SQL Pool (formerly SQL DW)
Hi all , Here is the scenario for which I am seeking suggestions: Whenever our data jobs run to refresh dimensions and facts, and a Tableau extract refresh occurs simultaneously, the extracts report inflated metrics. When we validate, the metrics on the…
0 answers
Connect and query to Azure database for PostgresSql from Synapse Analytics
Hi, Is there a way to connect and query Azure Database for PostgresSql from Synapse Analytics.We need near real time data from postgres database to join the CosmosDB data in Synapse Analytics. I would really appreciate for the input. Thanks!
asked 2024-06-13T12:33:24.1+00:00
Sathya Sai Cheetirala
0
Reputation points
commented 2024-06-17T04:44:29.6133333+00:00
Harishga
5,825
Reputation points Microsoft Vendor
0 answers
Changing the executor count in the Spark job definition configuration in Azure Synapse Analytics is not being reflected in the job
Hello All, Since my job was running for a long time, I did some analysis and decided to change the executor count to reduce the job running duration. My previous executor count was 4, and I tried to scale it up to 6. But the issue is, I could only…
asked 2024-06-07T07:02:11.77+00:00
Saravana Gopal
25
Reputation points
commented 2024-06-13T04:45:04.62+00:00
Smaran Thoomu
12,355
Reputation points Microsoft Vendor
0 answers
Apache Spark in Azure Synapse Analytics - HTTP request in notebook
Hello. I use a Notebook in Synapse where I run my Python code. I would like to make an API request from this Notebook to Microsoft Purview to send the entities. I added the pyapacheatlas library to spark. On my local computer, this code works fine…
0 answers
Synapse notebook connect to Azure SQL Database
Greetings!!! I am working on connecting to Azure SQL Database from an Azure Synapse analytics notebook using Azure Active Directory Integrated authentication using code below. Code: import requests import…
asked 2021-10-04T23:25:02.367+00:00
Raj D
586
Reputation points
commented 2024-06-10T21:01:03.9+00:00
Gnaneswar Reddy Sodini
0
Reputation points