Call databricks scripts from Aure data factory
Azure data factory is a wonderful tool to co-ordinate the entire EL and TL process. Using ADF, we can pull the data from the file stored in ADLS, FTP, web service, blob storage etc. transform the data using databricks/usql/hdinsight clusters and load into sqldb, azure sql dw, adls etc. Please go through the video below to understand how we can call databricks script from ADF and load the transformed output into Azure sql db.