Python ExternalMerger._recursive_merged_items示例

示例#1

文件： tests.py 项目： fireflyc/spark

 def test_huge_dataset(self):
     m = ExternalMerger(self.agg, 10)
     m.mergeCombiners(map(lambda (k, v): (k, [str(v)]), self.data * 10))
     self.assertTrue(m.spills >= 1)
     self.assertEqual(sum(len(v) for k, v in m._recursive_merged_items(0)),
             self.N * 10)
     m._cleanup()

示例#2

文件： tests.py 项目： lizhanyang505/spark

 def test_huge_dataset(self):
     m = ExternalMerger(self.agg, 10)
     m.mergeCombiners(map(lambda (k, v): (k, [str(v)]), self.data * 10))
     self.assertTrue(m.spills >= 1)
     self.assertEqual(sum(len(v) for k, v in m._recursive_merged_items(0)),
                      self.N * 10)
     m._cleanup()