This post I would like to show you how we can fix the problem of " Incorrect header check " received while fetching the data from hive table. Actual message " SparkException: Job aborted due to stage failure: Task 0 in stage 63.0 failed 4 times, most recent failure: Lost task 0.3 in stage 63.0 (TID 3506, 10.1.1.7, executor 28): java.io.IOException: incorrect header check. at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.inflateBytesDirect(Native Method) at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.decompress(ZlibDecompressor.java:221) at org.apache.hadoop.io.compress.DecompressorStream.decompress(DecompressorStream.java:82) at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:76) at java.io.InputStream.read(InputStream.java:101) " To better understand the scenario, let me explain how the data got loaded in the hive table. we are storing the data from source systems to RAW layer. H
Choose Newborn Photographer Perth Channon Williamson?
ReplyDeleteThis is really informative blog, I have to thank for your efforts. Waiting for more post like this.
ReplyDeleteBig Data Analytics Courses in Chennai
Data Analytics Training in Bangalore
Data Analyst course in Coimbatore
Hadoop Administration Training in Chennai
Salesforce Programming Skills
Great Post!!! thanks for sharing.
ReplyDeleteWhy is Big Data Important?
How to Learn Big Data?
With a well-defined schema, DataFrame 21 facilitates effective data handling, transformations, and querying for various data-driven tasks. Why No Game It is important to learn schema.
ReplyDelete