def cleaningAll(): #on lit le fichier info comme une liste dans laquelle chaque element correspond a une ligne (delimiter=||| jamais trouve), on divise ensuite la ligne 24 qui correspond a la liste des processus pour avoir le nombre total de processus number_process=len(loadtxt(os.path.join(path,project_name,"info.csv"), delimiter="|||", dtype=str)[24].split(",")) for simulation in [ e[18:] for e in os.listdir(os.path.join(path,project_name)) if e[:18]=="correlated_impacts" and e<>"correlated_impacts"]: number_iteration_fini=len(loadtxt(os.path.join(path,project_name,"correlated_impacts"+str(simulation),"0.csv"), delimiter="|", dtype=str))-1 for i in range(number_process): cleaning(i,simulation,number_iteration_fini, "correlated_impacts") for essai in [ e for e in os.listdir(os.path.join(path,project_name)) if e[:18]=="correlated_impacts" and e<>"correlated_impacts"]: os.rmdir(os.path.join(path,project_name,essai)) for simulation in [ e[20:] for e in os.listdir(os.path.join(path,project_name)) if e[:20]=="nocorrelated_impacts" and e<>"nocorrelated_impacts"]: for i in range(number_process): try: cleaning(i,simulation,False, "nocorrelated_impacts") except: 1 for essai in [ e for e in os.listdir(os.path.join(path,project_name)) if e[:20]=="nocorrelated_impacts" and e<>"nocorrelated_impacts"]: os.rmdir(os.path.join(path,project_name,essai)) UP_list=loadtxt(os.path.join(path,project_name,"info.csv"), delimiter="|||", dtype=str)[25][1:-1].split("\",\"") CF_categories=[name[:name.find(" (")] for name in loadtxt(os.path.join(path,project_name,"info.csv"), delimiter="|||", dtype=str)[6][1:-1].split(" , ")] impact_method=loadtxt(os.path.join(path,project_name,"info.csv"), delimiter="|||", dtype=str)[4][17:] sigma_correlated, mu_correlated, sign_correlated=calcul_parameters(UP_list, os.path.join(path,project_name,"correlated_impacts"), len(CF_categories)) results_cor = csv.writer(open(os.path.join(path,project_name,"Monte-Carlo_results_correle.csv"), "wb")) results_cor.writerow(["index", "processus"]+["sign "+category for category in CF_categories]+["mu "+category for category in CF_categories]+["sigma "+category for category in CF_categories]) for up in range(len(UP_list)-4): results_cor.writerow([up, UP_list[up]]+[ssign for ssign in sign_correlated[up,:].tolist()]+[mmu for mmu in mu_correlated[up,:].tolist()]+[sigma for sigma in sigma_correlated[up,:].tolist()]) sigma_nocorrelated, mu_nocorrelated, sign_nocorrelated=calcul_parameters(UP_list, os.path.join(path,project_name,"nocorrelated_impacts"), len(CF_categories)) results_nocor = csv.writer(open(os.path.join(path,project_name,"Monte-Carlo_results_nocorrele.csv"), "wb")) results_nocor.writerow(["index", "processus"]+["sign "+category for category in CF_categories]+["mu "+category for category in CF_categories]+["sigma "+category for category in CF_categories]) for up in range(len(UP_list)-4): try: results_nocor.writerow([up, UP_list[up]]+[ssign for ssign in sign_nocorrelated[up,:].tolist()]+[mmu for mmu in mu_nocorrelated[up,:].tolist()]+[sigma for sigma in sigma_nocorrelated[up,:].tolist()]) except: results_nocor.writerow([up, UP_list[up]]) ###Calculation of the cariance-covariance matrices calcul_vcv(UP_list, impact_method, CF_categories, os.path.join(path,project_name))
def calculExecution(): ''' Construction of technology and intervention matrices with uncertainties informations Construction of Caracterisation facteurs matrix ''' runButton.config(state="disabled") global pb_hD system_filename = os.path.join(path, "programme", "..", "databases", database) #export from Simapro infoFrame1 = Frame(informationsFrame) infoFrame1.pack() Label(infoFrame1, text="Reading the database and constructing the matrices...").pack( side=LEFT) (system_meta_info, UP_meta_info, UP_list, EF_list, all_flow, technology_matrix, intervention_matrix, CF_matrices, CF_categories, CF_units, EF_unit, unit_converter, infrastructure_rescale, uncertainty_info) = SimaPro_reader(system_filename, impact_method) Label(infoFrame1, text="Done").pack() CF_matrix = CF_matrices[impact_method] CF_categories = CF_categories[impact_method] CF_units = CF_units[impact_method] CF_categories_name = [re.sub("\W", "_", cat) for cat in CF_categories] EF_by_number = {} for (compartment, substance, subcompartment) in EF_list: EF = [compartment, substance, subcompartment] EF_number = EF_list.index(EF) EF = (compartment, substance, subcompartment) EF_by_number[EF_number] = EF ###Transformation of the technology and intervention matrices for testing the fonctionnality of the algorithme. infoFrame2 = Frame(informationsFrame) infoFrame2.pack() Label(infoFrame2, text="transformating matrices for tests ...").pack(side=LEFT) (technology_matrix, intervention_matrix, uncertainty_info, UP_list, CF_transformed) = transformation_matrices(technology_matrix, intervention_matrix, uncertainty_info, UP_list, CF_matrix) Label(infoFrame2, text="Done").pack() print_results(path + project_name, project_name, UP_list, EF_list, database, impact_method, CF_categories, CF_units, iterations, disaggregation_criterion, uncertainty_info, CF_matrix) ###Calculation of the determinists scores infoFrame3 = Frame(informationsFrame) infoFrame3.pack() Label(infoFrame3, text="Calculating deterministic scores ...").pack(side=LEFT) t0 = time.time() #inverse_technology_matrix=spsolve(technology_matrix, identity(technology_matrix.shape[0])) inverse_technology_matrix = inv(technology_matrix.todense()) tinv = time.time() - t0 #print "temps d'inversion : "+str(tinv) Z = (identity(len(technology_matrix.todense())) - technology_matrix) inverse_technology_matrix = fix_inverse(Z, inverse_technology_matrix) intensity_matrix = matrix( intervention_matrix.dot(inverse_technology_matrix)) all_system_scores, all_unit_scores = calculate_all_scores( identity(len(technology_matrix.todense())), intensity_matrix, intervention_matrix, CF_matrix) Label(infoFrame3, text="Done").pack() if correlatedMC.get(): ###Monte-Carlo in the correlated case and storage of the matrices (laws'parameters) Label( informationsFrame, text="Uncertainty analysis under a fully-correlated assumption..." ).pack() pb_hD.pack() essai = 0 while 1: try: os.mkdir( os.path.join(path, project_name, "correlated_impacts" + str(essai))) break except: essai += 1 (variables_technologique, variables_intervention) = MC_correlated_preparation( technology_matrix, intervention_matrix, uncertainty_info['technology'], uncertainty_info['intervention']) MC(variables_technologique, variables_intervention, CF_matrix, CF_categories_name, iterations, UP_list, "all", os.path.join(path, project_name, "correlated_impacts" + str(essai)), [], progress) infoFrame4 = Frame(informationsFrame) infoFrame4.pack() Label(infoFrame4, text="Printing parameters and covariances ...").pack(side=LEFT) sigma_correlated, mu_correlated, sign_correlated = calcul_parameters( UP_list, os.path.join(path, project_name, "correlated_impacts"), len(CF_categories)) results_cor = csv.writer( open( os.path.join(path, project_name, "Monte-Carlo_results_correle.csv"), "wb")) results_cor.writerow( ["index", "processus"] + ["impact " + category for category in CF_categories] + ["sign " + category for category in CF_categories] + ["mu " + category for category in CF_categories] + ["sigma " + category for category in CF_categories]) for up in range(len(UP_list) - 4): results_cor.writerow( [up, UP_list[up]] + [impact[0] for impact in all_system_scores[:, up].tolist()] + [ssign[0] for ssign in sign_correlated[:, up].tolist()] + [mmu[0] for mmu in mu_correlated[:, up].tolist()] + [sigma[0] for sigma in sigma_correlated[:, up].tolist()]) ###Calculation of the cariance-covariance matrices calcul_vcv(UP_list, impact_method, CF_categories_name, os.path.join(path, project_name)) Label(infoFrame4, text="Done").pack() if nocorrelatedMC.get(): sigma_correlated, mu_correlated, sign_correlated = calcul_parameters( UP_list, os.path.join(path, project_name, "correlated_impacts"), len(CF_categories)) essai = 0 while 1: try: os.mkdir( os.path.join(path, project_name, "nocorrelated_impacts" + str(essai))) break except: essai += 1 ###Monte-Carlo in the correlated case and storage of the matrices (laws'parameters) Label( informationsFrame, text="Uncertainty analysis under a fully-uncorrelated assumption..." ).pack() full_results_UP = {} full_results_EF = {} level_reached = {} system_scores = {} child_list = {} score_list_EF = {} coefficient_list = {} link_UP_EF_full_result = {} systems = [] for proc in UP_list: systems.append({proc: 1}) processRunned = Label(informationsFrame, text="") processRunned.pack() for system_number in range(nocorrBegin.get(), max(nocorrEnd.get(), len(UP_list) - 4)): #disaggregation for every system processRunned.config(text="Process " + str(system_number)) pb_hD.pack() full_results_UP = {} full_results_EF = {} level_reached = {} system_scores = {} child_list = {} score_list_EF = {} coefficient_list = {} link_UP_EF_full_result = {} final_demand_vector = build_final_demand_vector( systems[system_number], UP_list) start_time = time.time() full_results_UP = {} level_reached = {} #Desagregation of the system full_results_UP, level_reached, system_scores = systematic_disaggregation_UP( disaggregation_criterion, full_results_UP, level_reached, system_scores, UP_meta_info, UP_list, EF_list, technology_matrix, intervention_matrix, CF_matrix, CF_categories, EF_unit, uncertainty_info, intensity_matrix, Z, all_system_scores, all_unit_scores, impact_method, final_demand_vector, system_number, systems) UP_list_desag = construct_UP_list_desag(full_results_UP, UP_list) tree(UP_list_desag, UP_meta_info, impact_method, CF_categories, all_system_scores, all_unit_scores, CF_units, os.path.join(path, project_name, "trees")) (variables_technologique, variables_intervention) = MC_nocorrelated_preparation( technology_matrix, intervention_matrix, uncertainty_info['technology'], uncertainty_info['intervention'], UP_list, UP_list_desag, mu_correlated, sign_correlated, sigma_correlated) MC( variables_technologique, variables_intervention, CF_transformed, CF_categories_name, iterations, UP_list_desag, system_number, os.path.join(path, project_name, "nocorrelated_impacts" + str(essai)), systems, progress) infoFrame5 = Frame(informationsFrame) infoFrame5.pack() Label(infoFrame5, text="Printing parameters ...").pack(side=LEFT) sigma_nocorrelated, mu_nocorrelated, sign_nocorrelated = calcul_parameters( UP_list[:-4], os.path.join(path, project_name, "correlated_impacts"), len(CF_categories)) results_nocor = csv.writer( open( os.path.join(path, project_name, "Monte-Carlo_results_nocorrele.csv"), "wb")) results_nocor.writerow( ["index", "processus"] + ["impact " + category for category in CF_categories] + ["sign " + category for category in CF_categories] + ["mu " + category for category in CF_categories] + ["sigma " + category for category in CF_categories]) for up in range(len(UP_list) - 4): results_nocor.writerow( [up, UP_list[up]] + [impact[0] for impact in all_system_scores[:, up].tolist()] + [ssign[0] for ssign in sign_nocorrelated[:, up].tolist()] + [mmu[0] for mmu in mu_nocorrelated[:, up].tolist()] + [sigma[0] for sigma in sigma_nocorrelated[:, up].tolist()]) Label(infoFrame5, text="Done").pack(side=LEFT)
def calculExecution(): ''' Construction of technology and intervention matrices with uncertainties informations Construction of Caracterisation facteurs matrix ''' runButton.config(state="disabled") global pb_hD system_filename = os.path.join(path,"programme","..","databases",database) #export from Simapro infoFrame1=Frame(informationsFrame) infoFrame1.pack() Label(infoFrame1,text="Reading the database and constructing the matrices...").pack(side=LEFT) (system_meta_info, UP_meta_info, UP_list, EF_list,all_flow, technology_matrix, intervention_matrix, CF_matrices, CF_categories, CF_units, EF_unit, unit_converter, infrastructure_rescale, uncertainty_info) = SimaPro_reader(system_filename, impact_method) Label(infoFrame1,text="Done").pack() CF_matrix = CF_matrices[impact_method] CF_categories=CF_categories[impact_method] CF_units=CF_units[impact_method] CF_categories_name=[ re.sub("\W","_",cat) for cat in CF_categories] EF_by_number = {} for (compartment, substance, subcompartment) in EF_list: EF = [compartment, substance, subcompartment] EF_number = EF_list.index(EF) EF = (compartment, substance, subcompartment) EF_by_number[EF_number] = EF ###Transformation of the technology and intervention matrices for testing the fonctionnality of the algorithme. infoFrame2=Frame(informationsFrame) infoFrame2.pack() Label(infoFrame2,text="transformating matrices for tests ...").pack(side=LEFT) (technology_matrix,intervention_matrix,uncertainty_info, UP_list, CF_transformed)=transformation_matrices(technology_matrix,intervention_matrix,uncertainty_info, UP_list, CF_matrix) Label(infoFrame2,text="Done").pack() print_results(path+project_name, project_name, UP_list, EF_list, database, impact_method, CF_categories, CF_units, iterations, disaggregation_criterion, uncertainty_info, CF_matrix) ###Calculation of the determinists scores infoFrame3=Frame(informationsFrame) infoFrame3.pack() Label(infoFrame3,text="Calculating deterministic scores ...").pack(side=LEFT) t0=time.time() #inverse_technology_matrix=spsolve(technology_matrix, identity(technology_matrix.shape[0])) inverse_technology_matrix = inv(technology_matrix.todense()) tinv=time.time()-t0 #print "temps d'inversion : "+str(tinv) Z = (identity(len(technology_matrix.todense())) - technology_matrix) inverse_technology_matrix = fix_inverse(Z, inverse_technology_matrix) intensity_matrix = matrix(intervention_matrix.dot(inverse_technology_matrix)) all_system_scores, all_unit_scores = calculate_all_scores(identity(len(technology_matrix.todense())),intensity_matrix, intervention_matrix, CF_matrix) Label(infoFrame3,text="Done").pack() if correlatedMC.get(): ###Monte-Carlo in the correlated case and storage of the matrices (laws'parameters) Label(informationsFrame,text="Uncertainty analysis under a fully-correlated assumption...").pack() pb_hD.pack() essai=0 while 1: try: os.mkdir(os.path.join(path,project_name,"correlated_impacts"+str(essai))) break except: essai+=1 (variables_technologique, variables_intervention)=MC_correlated_preparation(technology_matrix, intervention_matrix, uncertainty_info['technology'], uncertainty_info['intervention']) MC(variables_technologique, variables_intervention, CF_matrix, CF_categories_name, iterations, UP_list, "all", os.path.join(path,project_name,"correlated_impacts"+str(essai)), [], progress) infoFrame4=Frame(informationsFrame) infoFrame4.pack() Label(infoFrame4,text="Printing parameters and covariances ...").pack(side=LEFT) sigma_correlated, mu_correlated, sign_correlated=calcul_parameters(UP_list, os.path.join(path,project_name,"correlated_impacts"), len(CF_categories)) results_cor = csv.writer(open(os.path.join(path,project_name,"Monte-Carlo_results_correle.csv"), "wb")) results_cor.writerow(["index", "processus"]+["impact "+category for category in CF_categories]+["sign "+category for category in CF_categories]+["mu "+category for category in CF_categories]+["sigma "+category for category in CF_categories]) for up in range(len(UP_list)-4): results_cor.writerow([up, UP_list[up]]+[impact[0] for impact in all_system_scores[:,up].tolist()]+[ssign[0] for ssign in sign_correlated[:,up].tolist()]+[mmu[0] for mmu in mu_correlated[:,up].tolist()]+[sigma[0] for sigma in sigma_correlated[:,up].tolist()]) ###Calculation of the cariance-covariance matrices calcul_vcv(UP_list, impact_method, CF_categories_name, os.path.join(path,project_name)) Label(infoFrame4,text="Done").pack() if nocorrelatedMC.get(): sigma_correlated, mu_correlated, sign_correlated=calcul_parameters(UP_list, os.path.join(path,project_name,"correlated_impacts"), len(CF_categories)) essai=0 while 1: try: os.mkdir(os.path.join(path,project_name,"nocorrelated_impacts"+str(essai))) break except: essai+=1 ###Monte-Carlo in the correlated case and storage of the matrices (laws'parameters) Label(informationsFrame,text="Uncertainty analysis under a fully-uncorrelated assumption...").pack() full_results_UP = {} full_results_EF = {} level_reached = {} system_scores = {} child_list = {} score_list_EF = {} coefficient_list = {} link_UP_EF_full_result = {} systems=[] for proc in UP_list: systems.append({proc:1}) processRunned=Label(informationsFrame,text="") processRunned.pack() for system_number in range(nocorrBegin.get(),max(nocorrEnd.get(),len(UP_list)-4)): #disaggregation for every system processRunned.config(text="Process "+str(system_number)) pb_hD.pack() full_results_UP = {} full_results_EF = {} level_reached = {} system_scores = {} child_list = {} score_list_EF = {} coefficient_list = {} link_UP_EF_full_result = {} final_demand_vector = build_final_demand_vector(systems[system_number], UP_list) start_time = time.time() full_results_UP = {} level_reached = {} #Desagregation of the system full_results_UP, level_reached, system_scores = systematic_disaggregation_UP(disaggregation_criterion,full_results_UP, level_reached, system_scores,UP_meta_info, UP_list, EF_list, technology_matrix, intervention_matrix, CF_matrix, CF_categories, EF_unit, uncertainty_info, intensity_matrix, Z, all_system_scores, all_unit_scores, impact_method, final_demand_vector, system_number,systems) UP_list_desag=construct_UP_list_desag(full_results_UP,UP_list) tree(UP_list_desag, UP_meta_info, impact_method, CF_categories, all_system_scores, all_unit_scores, CF_units, os.path.join(path,project_name,"trees")) (variables_technologique, variables_intervention)=MC_nocorrelated_preparation(technology_matrix, intervention_matrix, uncertainty_info['technology'], uncertainty_info['intervention'], UP_list, UP_list_desag, mu_correlated, sign_correlated, sigma_correlated) MC(variables_technologique, variables_intervention, CF_transformed, CF_categories_name, iterations, UP_list_desag, system_number, os.path.join(path,project_name,"nocorrelated_impacts"+str(essai)), systems, progress) infoFrame5=Frame(informationsFrame) infoFrame5.pack() Label(infoFrame5,text="Printing parameters ...").pack(side=LEFT) sigma_nocorrelated, mu_nocorrelated, sign_nocorrelated=calcul_parameters(UP_list[:-4], os.path.join(path,project_name,"correlated_impacts"), len(CF_categories)) results_nocor = csv.writer(open(os.path.join(path,project_name,"Monte-Carlo_results_nocorrele.csv"), "wb")) results_nocor.writerow(["index", "processus"]+["impact "+category for category in CF_categories]+["sign "+category for category in CF_categories]+["mu "+category for category in CF_categories]+["sigma "+category for category in CF_categories]) for up in range(len(UP_list)-4): results_nocor.writerow([up, UP_list[up]]+[impact[0] for impact in all_system_scores[:,up].tolist()]+[ssign[0] for ssign in sign_nocorrelated[:,up].tolist()]+[mmu[0] for mmu in mu_nocorrelated[:,up].tolist()]+[sigma[0] for sigma in sigma_nocorrelated[:,up].tolist()]) Label(infoFrame5,text="Done").pack(side=LEFT)