Python RDD.persistの例

コード例 #1

 def __call__(self, head: RDD):
     if self.head is None or self.trace != self.path():
         self.head = head.persist(self.persistence)
         self.trace = self.path()
     return self.head

コード例 #2

def materialize(rdd: RDD):
    rdd.persist(STORAGE_LEVEL)
    rdd.mapPartitionsWithIndex(lambda ind, it: (1, )).collect()
    return rdd