Azure Data lake - Query

I was going through data lake job creation and re usability of the scripts written. Important point I would like to point out is that in data lake we cannot go back and correct any mistake in the query created or any modification in the existing query but have to create new query. I believe this has been done to track the changes in each and every query.

It is easy to create any new query by selecting the sample queries provided by Microsoft and change the syntax and table names based on our new requirements and finally saving with different names.

Sample query to extract the records from file.

@Searchlog =
 Extract         UserID              int,
                      Start                  Datetime,
                     Region               String,
                     Query                String,
                     Duration            int,
                     Urls                   String,
                     ClickedUrls       String
 FROM @"/Samples/Data/Searchlog.tsv"   -- File from where we need to fetch records
 USING Extractors.Tsv();  -- Extractors are predefined expressions to extract records

-- below statement move the extracted records into another file
 OUTPUT @Searchlog  
 TO @"/Samples/Data/Searchlog_output.tsv"
 USING Outputers.Tsv();


Popular posts from this blog

Hadoop - Hive - Load data from csv/xls files

Microsoft BI Implementation - Cube back up and restore using XMLA command