PySpark Distinct List of Each of the Keys from an RDD

Viewed 12680

I'm sure this is simple, but I keep having issues. I have an RDD with key value pairs. I want a distinct list of just the keys. I'll share the code and examples. Thank you in advance!

RDD Example

>>> rdd4.take(3)
[[(u'11394071', 1), (u'11052103', 1), (u'11052101', 1)], [(u'11847272', 10), (u'999999', 1), (u'11847272', 10)], [(u'af1lowprm1704', 5), (u'am1prm17', 2), (u'af1highprm1704', 2)]]

Tried / Didn't Work

rdd4.distinct().keys()
rdd4.map(lambda x: tuple(sorted(x))).keys().distinct()

[(u'10972402', 1), (u'10716707', 1), (u'11165362', 1)]

Preferred Structure

[u'11394071', u'11052101', '999999', u'11847272', u'am1prm17', u'af1highprm1704']
3 Answers
Related