Thursday, April 30, 2015

Data localization in mapreduce

Whenever the mapper process is completed before tasktracker emits the result . Task tracker keep the output in the LFS(Local file system) in the same node.

Note : - Data localization is for mapper phase but not for sort & shuffle and reducer phase.

Life of mapper output is till the end of job completion i.e as the job completion success or failure the local copies of mapper o/p will automatically revoked by mapper only.


No comments:

Post a Comment