I have a huge datasets with MM+ records and I am trying to assign unique id to each record. I tried below code but it takes lot of time as row id is sequential. I have tried tweak memory parameters to optimize job, couldn't gain much performance.
sample snippet:
JavaRDD<String> rawRdd=......
rawRdd.zipWithIndex()
.mapToPair(t->new Tuple2<Long,String>(t._2,t._1))
Are there any better way to assign unique id? thanks