4,653 questions with Azure Synapse Analytics tags

Sort by: Updated
Updated Created Answers
1 answer One of the answers was accepted by the question author.

Can I create external views in synapse from multiple data sources?

Hi, I have data from different azure regions and my idea to decrease overhead is to use just synapse workspace with multiple external data sources and create the external tables for each of them, since I can't put all the data into a same location for…

Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,897 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
asked 2024-07-10T00:58:48.06+00:00
Gustavo Adolfo Alemán Sánchez 60 Reputation points
accepted 2024-07-10T06:19:24.84+00:00
Gustavo Adolfo Alemán Sánchez 60 Reputation points
1 answer One of the answers was accepted by the question author.

Static Code Analysis for ADF or Synapse Pipelines in Azure

Hi team, I would like to know is there any use case possible to perform static code analysis of our ADF or Synapse pipelines residing in Azure DevOps repository? Thanks, Debajit Mitra

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,078 questions
asked 2024-06-14T03:42:29.93+00:00
Debajit Mitra 20 Reputation points
accepted 2024-07-10T04:04:11.9866667+00:00
Debajit Mitra 20 Reputation points
4 answers

unable to delete the Apache Spark pool in Azure Synapse Analytics

unable to delete the Apache Spark pool and attached error messages. I tried to delete via powershell as well from Azure Cloud Shell but no luck. As you can see in the screenshot there is no notebook1.

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
asked 2023-02-08T18:28:02.17+00:00
Karthik R 5 Reputation points
commented 2024-07-09T15:59:50.1733333+00:00
Andreas Schiffler 0 Reputation points Microsoft Employee
1 answer

merge statement in 2 data frame

Hi , how to use merge statement in 2 dataframe df1=spark.sql("" sellect cole1,col2 from table1""") df2=spark.sql("" sellect cole1,col2 from table2""") expected results merge into table2 using tabl1 on…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,059 questions
asked 2024-07-04T19:49:06.46+00:00
Vineet S 325 Reputation points
commented 2024-07-09T10:53:09.3966667+00:00
Smaran Thoomu 12,355 Reputation points Microsoft Vendor
1 answer

Issues Connecting to Infor Data Lake Using Spark JDBC in Azure Synapse Spark Notebook

Hi Team, I'm encountering difficulties while attempting to connect to Infor Data Lake using Spark JDBC within Azure Synapse. Despite configuring the necessary JDBC driver and ionapi file path, I consistently encounter the following…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
asked 2024-07-04T09:19:18.87+00:00
Sriram M 20 Reputation points
commented 2024-07-09T10:19:01.5433333+00:00
Harishga 5,825 Reputation points Microsoft Vendor
1 answer

multiple lookup in copy activity

Hi, how to handle multiple lookup column as input in copy activity

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,078 questions
asked 2024-07-06T02:27:07.44+00:00
Vineet S 325 Reputation points
commented 2024-07-09T08:59:00.3+00:00
Chandra Boorla 155 Reputation points Microsoft Vendor
2 answers One of the answers was accepted by the question author.

Extracting Purchaser Data from Azure Synapse for Marketing Automation

I'm looking to connect Azure Synapse Analytics with a marketing automation platform. My goal is to identify recent purchasers from my existing Azure Synapse data and send them targeted emails using the marketing platform. While I can handle sending…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
asked 2024-07-04T09:11:47.34+00:00
vikranth-0706 200 Reputation points
accepted 2024-07-09T05:17:32.1633333+00:00
vikranth-0706 200 Reputation points
1 answer

AZ DATA FACTORY - COPY Activity : 'Incorrect syntax near '>'

Hi, while i am trying to copy data using below dynamic value. SELECT * FROM @{pipeline().parameters.P_DS_Object_api} WHERE lastmodifieddate >= '@{formatDateTime(activity('watermarktable').output.firstRow.Watermarkvalue, 'yyyy-MM-dd HH:mm:ss')}' i am…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,078 questions
asked 2023-09-07T11:12:25.0733333+00:00
Ram Babu 50 Reputation points
commented 2024-07-09T03:55:13.32+00:00
Shweta Sapate 0 Reputation points
1 answer

Notification email fail/not started pipeline Synapse

I would like to set up a notification (email) for a pipeline in Synapse to be sent if it fails or if it does not start. How do I do this? I have already created an Azure alert, but it only triggers if the pipeline encounters an error, not if it does not…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
asked 2024-06-27T09:18:56.04+00:00
MG 5 Reputation points
commented 2024-07-08T22:01:18.8233333+00:00
BhargavaGunnam-MSFT 28,931 Reputation points Microsoft Employee
1 answer

Guidance on how to use Service Principal with Certificate to Authorize for EventHub Stream Read

I found this documentation https://github.com/Azure/azure-event-hubs-spark/blob/master/docs/use-aad-authentication-to-connect-eventhubs.md online on how to use service principal with certificate to use spark stream read from EventHubs, I want to do this…

Azure Event Hubs
Azure Event Hubs
An Azure real-time data ingestion service.
593 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,059 questions
Microsoft Entra ID
Microsoft Entra ID
A Microsoft Entra identity service that provides identity management and access control capabilities. Replaces Azure Active Directory.
20,460 questions
asked 2024-07-01T21:27:20.97+00:00
BEPV 0 Reputation points
commented 2024-07-08T18:29:43.01+00:00
BEPV 0 Reputation points
2 answers One of the answers was accepted by the question author.

Azure Synapse LinkConnection template json

We have created a LinkConnection in Azure synapse, the source database is MS SQL Server and target database is Dedicated SQL Pool. We are trying to create powershell script to automate the creation of this LinkConnection for each environment - Testing,…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
Azure Automation
Azure Automation
An Azure service that is used to automate, configure, and install updates across hybrid environments.
1,184 questions
PowerShell
PowerShell
A family of Microsoft task automation and configuration management frameworks consisting of a command-line shell and associated scripting language.
2,288 questions
asked 2024-07-03T19:21:58.6166667+00:00
Girish Prabhudesai 41 Reputation points
accepted 2024-07-08T11:30:36.9633333+00:00
Girish Prabhudesai 41 Reputation points
1 answer One of the answers was accepted by the question author.

How to install com.crealytics.spark.excel to Synapse Spark Pool

Hello, I have spark pool with following properties: According to the instruction mentioned under the Q&A https://learn.microsoft.com/en-us/answers/questions/1185314/how-to-write-data-from-notebook-to-dedicated-sql-p?cid=kerryherger I have…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
asked 2024-06-26T14:34:57.57+00:00
Tamashevich, Tatsiana 60 Reputation points
accepted 2024-07-08T10:20:43.0566667+00:00
Tamashevich, Tatsiana 60 Reputation points
1 answer

Azure Synapse CETAS is not working

Hi, When creating an external table in Synapse SQL Pools (SSP) using the following query, I encountered an error: "Operation failed since the external data source 'https://<storageacc>.dfs.core.windows.net/<containername>/asa' has…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
asked 2024-07-08T01:33:11.29+00:00
Amila Akalanka 0 Reputation points
answered 2024-07-08T09:24:29.7033333+00:00
phemanth 8,480 Reputation points Microsoft Vendor
2 answers One of the answers was accepted by the question author.

Azure Synaps serverless scan file that should be filtered

Hey, We have a large amount of files stored in Storage Account in Parquet format with the following hierarchy. /m_id=%M_ID%/quarter=%QUARTER%/month=%MONTH%/partition_version=last/part-0.parquet'. examples for quarter values: 2024Q1, 2024Q2, 2024Q3…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
asked 2024-07-01T08:48:06.6533333+00:00
Shlomi Lanton 76 Reputation points
accepted 2024-07-08T06:32:50.6833333+00:00
Shlomi Lanton 76 Reputation points
2 answers

This request is not authorized to perform this operation using this permission.", 403, HEAD Synapse connect to adls

I am trying to select data from ADLS Gen2 storage delta table and keep receiving this error. I added the synapse service principal as storage blob data contributor and ACLs to container with no luck. Firewall is set to enable all networks as well. Please…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,418 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
asked 2024-06-22T20:29:54.0233333+00:00
Daniel Woodhouse 0 Reputation points
commented 2024-07-08T05:18:52.7933333+00:00
Nehruji R 4,376 Reputation points Microsoft Vendor
1 answer

Import filenames from Azure Storage container to SQL Table using Azure Data Factory

Hi All, I am trying to import filenames from an Azure Storage container into a table in a Dedicated SQL Pool (formerly SQL DW) using Azure Data Factory. For example, I have around 2,000 JSON files in an Azure Storage container, and I would like to…

Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,897 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,599 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,078 questions
asked 2024-06-27T01:33:27.9933333+00:00
PS 396 Reputation points
commented 2024-07-08T05:16:58.4033333+00:00
Nehruji R 4,376 Reputation points Microsoft Vendor
1 answer

unable connect Azure Synapse database

Hi, while creating link services for connecting azure synapse database..i am getting error that Insufficient permissions to call this API. 1829be47-97cd-4a80-b8f7-9402f8fbcc20 does not have Microsoft.Synapse/workspaces/credentials/useSecret/action on…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,078 questions
asked 2024-07-01T19:38:25.9533333+00:00
Vineet S 325 Reputation points
commented 2024-07-08T02:51:59.0266667+00:00
PRADEEPCHEEKATLA-MSFT 84,936 Reputation points Microsoft Employee
1 answer

Unable to run an SQL query in Azure Synapse

Hi, I am unbale to run a simple sql query. It keeps throwing this following error. File 'https://storagecontainer/Financial Sample.xlsx' cannot be opened because it does not exist or it is used by another process. I assigned myself with Storage Blob Data…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
asked 2024-06-30T14:25:48.4233333+00:00
shailaja Azure 0 Reputation points
edited a comment 2024-07-08T02:51:08.58+00:00
PRADEEPCHEEKATLA-MSFT 84,936 Reputation points Microsoft Employee
1 answer One of the answers was accepted by the question author.

Azure Api version retirements and resource arm versions doubt

Due to the breaking change "API version retirements (September 2023)" we are working on update all of our ARM templates to use API version 2021-08-01 or later as Microsoft indicates in documentation. But there are some resources which their…

Azure API Management
Azure API Management
An Azure service that provides a hybrid, multi-cloud management platform for APIs.
1,930 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
asked 2023-04-03T08:11:15.25+00:00
PATRICIA FUENTES 51 Reputation points
commented 2024-07-07T11:43:46.8733333+00:00
shani 0 Reputation points
2 answers One of the answers was accepted by the question author.

find character position of nth occurrence in a string in ADF

I have a field of string data and need to find nth occurrence of character position using ADF. example: instr('microsoft is good') first occurrence: 5 second occurrence: 7 third occurrence: 15 fourth occurrence: 16

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,078 questions
asked 2024-07-05T21:58:29.1233333+00:00
Peruka, Sikander reddy 90 Reputation points
answered 2024-07-07T08:23:05.94+00:00
Nandan Hegde 30,951 Reputation points MVP