def execute(self, sent): # 対話行為タイプの推定開始 # 4タイプ # - genre: イタリアンとか # - location: 新宿とか # - money: 1万円とか # - other: その他 features = sent2features_(sent) #print("送信されたテキストから変換したfeatures:", features) act_type = self.__predictor.predict([features]) print("featureから予測したact_type:", act_type) # 対話行為タイプの推定完了 # 属性抽出 # ジャンルに対して文字を抽出。イタリアンとか中華とか surfaces, features = analyze_morph(sent) print("surfaces:", surfaces) print("features:", features) morphed_sent = [[surfaces[i]] + features[i].split(',') for i in range(len(surfaces))] print("morphed_sent", morphed_sent) features = sent2features(morphed_sent) named_entity = self.__extractor.extract(features, morphed_sent) dialogue_act = {'user_act_type': act_type} # ここで属性を追加 dialogue_act.update(dict(named_entity)) return dialogue_act
def execute(self, sent): features = sent2features_(sent) # print("----------features_features--------------") # print(features) act_type = self.__predictor.predict([features]) # print("----------act_type--------------") # print(act_type) surfaces, features = analyze_morph(sent) # print("----------surfaces,features--------------") # print(features) # print(surfaces) morphed_sent = [[surfaces[i]] + features[i].split(',') for i in range(len(surfaces))] features = sent2features(morphed_sent) print("----------morphed_sent,features--------------") print(morphed_sent) print(features) named_entity = self.__extractor.extract(features, morphed_sent) print("----------named_entity--------------") print(named_entity) dialogue_act = {'user_act_type': act_type} if act_type != 'other': dialogue_act.update(dict(named_entity)) # # print("----------dialogue_act--------------") # print(dialogue_act) return dialogue_act
def execute(self, sent): features = sent2features_(sent) act_type = self.__predictor.predict([features]) surfaces, features = analyze_morph(sent) morphed_sent = [[surfaces[i]] + features[i].split(',') for i in range(len(surfaces))] features = sent2features(morphed_sent) named_entity = self.__extractor.extract(features, morphed_sent) dialogue_act = {'user_act_type': act_type} dialogue_act.update(dict(named_entity)) return dialogue_act
def execute(self, sent): features = sent2features_(sent) act_type = self.__predictor.predict([features]) surfaces, features = analyze_morph(sent) morphed_sent = [[surfaces[i]] + features[i].split(',') for i in range(len(surfaces))] features = sent2features(morphed_sent) named_entity = self.__extractor.extract(features, morphed_sent) dialogue_act = {'user_act_type': act_type} dialogue_act.update(dict(named_entity)) return dialogue_act
f = lambda path: os.path.dirname(path) root_dir = f(f(f(f(os.path.abspath(sys.argv[0]))))) training_data_dir = os.path.join(root_dir, 'training_data') training_data = [] for file_name in os.listdir(training_data_dir): if not file_name.endswith('.pkl'): continue file_path = os.path.join(training_data_dir, file_name) with open(file_path, 'rb') as rf: tmp_data = pickle.load(rf) training_data.extend(tmp_data) random.shuffle(training_data) train_num = int(len(training_data) * 0.9) train_sents = training_data[:train_num] test_sents = training_data[train_num:] train_x = [sent2features(s) for s in train_sents] train_y = [sent2labels(s) for s in train_sents] test_x = [sent2features(s) for s in test_sents] test_y = [sent2labels(s) for s in test_sents] extractor = NamedEntityExtractor() extractor.train(train_x, train_y) pred_y = [extractor.tagger(xseq) for xseq in test_x] print(extractor.evaluate(test_y, pred_y))
f = lambda path: os.path.dirname(path) root_dir = f(f(f(f(__file__)))) training_data_dir = os.path.join(root_dir, "training_data") training_data = [] for file_name in os.listdir(training_data_dir): if not file_name.endswith(".pkl"): continue file_path = os.path.join(training_data_dir, file_name) with open(file_path, "rb") as rf: tmp_data = pickle.load(rf) training_data.extend(tmp_data) random.shuffle(training_data) train_num = int(len(training_data) * 0.9) train_sents = training_data[:train_num] test_sents = training_data[train_num:] train_x = [sent2features(s) for s in train_sents] train_y = [sent2labels(s) for s in train_sents] test_x = [sent2features(s) for s in test_sents] test_y = [sent2labels(s) for s in test_sents] extractor = NamedEntityExtractor() extractor.train(train_x, train_y) pred_y = [extractor.tagger(xseq) for xseq in test_x] print(extractor.evaluate(test_y, pred_y))