Python from_file_parallel Exemples

Exemple #1

Fichier : backup-pagerank.py Projet : GatsbyNewton/graph-computation-benchmark

def loadMatrix(ctx, filePath, numPage):
    #return eager(expr.from_file(filePath, tile_hint = (numPage, numPage / 8)))
    return eager(expr.from_file_parallel(filePath, "numpy", sparse = True, tile_hint = (numPage, numPage/10)))

Exemple #2

Fichier : backup-bfs.py Projet : GatsbyNewton/graph-computation-benchmark

def loadMatrix(ctx, filePath, dim):
	return expr.from_file_parallel(filePath, "numpy", tile_hint=(dim, dim/ 10))

Exemple #3

Fichier : concomp.py Projet : GatsbyNewton/graph-computation-benchmark

def loadMatrix(ctx, filePath, dim):
	return expr.from_file_parallel(filePath, tile_hint = (dim, dim / ctx.num_workers))