def proceed(id): aliquot = Aliquot.query.get(id) for _item in range(len(aliquot.aliquot_items.all())): num = 0 _sample = aliquot.sample _sample.status = 3 sample = Sample() sample.code = _sample.code sample.parent_id = _sample.id sample.technique = _sample.technique sample.patient_id = _sample.patient_id sample.sample_nature_id = _sample.sample_nature_id sample.sample_type_id = _sample.sample_type_id sample.support_id = _sample.support_id sample.jonc_type_id = _sample.jonc_type_id sample.mesure_id = _sample.mesure_id sample.volume = aliquot.aliquot_items[num].volume sample.site = _sample.site sample.date = _sample.date sample.status = 0 aliquot.aliquot_items[num].status = 1 db.session.add(sample) db.session.commit() num = num + 1 generateCode(sample, aliquot.nbr_aliquot(), num) __sample = Sample.query.get(_sample.id) __sample.volume = 0 aliquot.status = 1 db.session.commit() flash(_('Enregistrement effectué avec succèss')) return redirect(url_for('aliquot.index'))
def add(): form = ProcessForm() form.sample.choices = [(c.id, c.code) for c in Basket.query.first().samples] if form.validate_on_submit(): process = Aliquot(sample_id=form.sample.data, number=form.number.data) db.session.add(process) db.session.commit() sample = Sample.query.get(form.sample.data) for x in range(int(form.number.data)): _sample = Sample() _sample.patient_id = sample.patient_id _sample.parent_id = sample.id _sample.code = sample.code + "-" + str(x) _sample.volume = int(sample.volume) / int(form.number.data) # _sample = Sample() # _sample.serial = get_bio_code() # _sample.code = s.code.data _sample.sample_type_id = sample.sample_type_id _sample.date = sample.date _sample.site = sample.site _sample.tube_type_id = sample.tube_type_id _sample.mesure_id = sample.mesure_id _sample.status = 0 _sample.in_basket = 0 _sample.basket_id = 0 db.session.add(_sample) sample.status = 1 db.session.commit() flash(_('Nouveau Client ajouté avec succèss!')) return redirect(url_for('process.detail', id=process.id)) return render_template('process/form.html', form=form)
def sample_init_func(row): p = Sample(code=row['code'], date=row['date'], jonc_type_id=row['jonc_type_id'], mesure_id=row['mesure_id'], parent_id=row['parent_id'], patient_id=row['patient_id'], sample_nature_id=row['sample_nature_id'], sample_type_id=row['sample_type_id'], status=row['status'], technique=row['technique'], support_id=row['support_id'], volume=row['volume'], origin_id=row['origin_id'], ) return p
def add(): form = SampleForm() form.patient.choices = [(c.id, c.code) for c in Patient.query.all()] form.sample_nature.choices = [(c.id, c.name) for c in SampleNature.query.all()] form.sample_type.choices = [(c.id, c.name) for c in SampleType.query.all()] form.support.choices = [(c.id, c.name) for c in Support.query.all()] form.jonc_type.choices = [(c.id, c.name) for c in JoncType.query.all()] form.mesure.choices = [(c.id, c.name) for c in Mesure.query.all()] if form.validate_on_submit(): for r in range(int(form.number.data)): print(form.number.data) sample = Sample() sample.code = form.code.data sample.technique = form.technique.data sample.results = form.results.data sample.patient_id = form.patient.data sample.sample_nature_id = form.sample_nature.data sample.sample_type_id = form.sample_type.data sample.support_id = form.support.data sample.jonc_type_id = form.jonc_type.data sample.mesure_id = form.mesure.data sample.volume = form.volume.data sample.site = form.site.data sample.number = form.number.data sample.date = form.date.data sample.status = 0 db.session.add(sample) db.session.commit() generateCode(sample, r + 1) flash(_('Nouveau prélèvement ajouté avec succèss!')) return redirect(url_for('sample.index')) return render_template('sample/form.html', form=form)
def update(): content = request.get_json(silent=False) all_feeds = content['all_feeds'] for feed in all_feeds: feed_name = feed['feed_name'] samples = feed['samples'] feed = Feed.query.filter_by(name=feed_name).first() if feed is None: feed = Feed(name=feed_name) db.session.add(feed) db.session.commit() for sample in samples: new_sample = Sample( feed_id=feed.id, value=sample['value'], time=datetime.fromtimestamp( sample['time']).replace(tzinfo=timezone.utc).astimezone( tz=local_tz).replace(tzinfo=None)) db.session.add(new_sample) db.session.commit() if len(Sample.query.all()) >= 9900: Thread(target=clean_history).start() return 'update success'
def add(order_id=0): order = None if order_id != 0: order = Order.query.get(order_id) form = PatientForm() form.order.choices = [(c.id, c.serial) for c in Order.query.all()] for entry in form.samples.entries: entry.sample_type.choices = [(c.id, c.name) for c in SampleType.query.all()] entry.mesure.choices = [(c.id, c.name) for c in Mesure.query.all()] entry.support.choices = [(c.id, c.name) for c in Support.query.all()] entry.sample_nature.choices = [(c.id, c.name) for c in SampleNature.query.all()] entry.jonc_type.choices = [(c.id, c.name) for c in JoncType.query.all()] if form.validate_on_submit(): patient = Patient(bio_code=get_bio_code('HU'), order_id=order_id, origin_id=1, code=form.code.data, sexe=form.sexe.data, birthday=form.birthday.data, age=form.age.data, city=form.city.data, job=form.job.data, clinical_data=form.clinical_data.data) db.session.add(patient) db.session.commit() for s in form.samples.entries: print(int(s.number.data)) for r in range(int(s.number.data)): sample = Sample() sample.origin_id = s.patient.origin_id sample.code = s.code.data sample.technique = s.technique.data sample.results = s.results.data sample.sample_nature_id = s.sample_nature.data sample.sample_type_id = s.sample_type.data sample.date = s.date.data sample.site = s.site.data sample.support_id = s.support.data sample.jonc_type_id = s.jonc_type.data sample.number = s.number.data sample.mesure_id = s.mesure.data sample.volume = s.volume.data sample.patient_id = patient.id sample.status = 0 db.session.add(sample) db.session.commit() generateCode(sample, r + 1) flash(_('Nouveau patient ajouté avec succèss!')) return redirect(url_for('patient.add', order_id=order_id)) form.order.data = order_id return render_template('patient/form.html', form=form, order=order)
def setUp(self): self.app = self.create_app() self.client = self.app.test_client() self.app_context = self.app.app_context() self.app_context.push() db.create_all() # Create some data to test against, but don't commit # Create a region maine = Region(name='Maine', slug='maine') db.session.add(maine) # Create a river andro = River(name='Androscoggin River', slug='androscoggin') wild = River(name='Wild River of the Androsoggin', slug='wild-androscoggin', parent=andro) db.session.add(andro) db.session.add(wild) # Create a section wild_section = Section(name='Wild River from Hastings', slug='wild-river', river=wild, putin='SRID=4326;POINT(-71.05996191501617 44.31595096222731)', takeout='SRID=4326;POINT(-70.97963511943817 44.390833083196924)', path='SRID=4326;LINESTRING(-71.05997800827026 44.316024368364864,-71.05881929397583 44.31798950769032,-71.05731725692749 44.31884923545022,-71.05444192886353 44.31851148676115,-71.05298280715942 44.31943261497028,-71.05096578598022 44.322687152160796,-71.05045080184937 44.32449856163325,-71.04984998703003 44.32495908054771,-71.04761838912964 44.325849406864485,-71.04568719863892 44.32649411748597,-71.04306936264038 44.32753791965937,-71.04049444198608 44.327138821021585,-71.03847742080688 44.32664761897048,-71.03607416152954 44.32572660403795,-71.03517293930054 44.32554239931617,-71.03431463241577 44.32594150881567,-71.03341341018677 44.32805981378074,-71.03264093399048 44.329103588092785,-71.02929353713989 44.32984035877702,-71.02843523025513 44.33008594694842,-71.02757692337036 44.33137526797721,-71.02804899215698 44.33309431859246,-71.02783441543579 44.33459844654652,-71.02543115615845 44.33582627750024,-71.02311372756958 44.33714616710255,-71.0210108757019 44.33926406740166,-71.01598978042603 44.34328479806066,-71.01294279098511 44.3441441551062,-71.00916624069214 44.346752840399844,-71.0082221031189 44.34972966884455,-71.00689172744751 44.35107993293547,-71.00551843643188 44.351171995263435,-71.00393056869507 44.350374116950185,-71.00148439407349 44.35000586175751,-70.9984803199768 44.350374116950185,-70.99642038345337 44.35163230473401,-70.99328756332397 44.354117913402796,-70.9923005104065 44.3552225945275,-70.99212884902954 44.35678752380721,-70.99401712417603 44.357830786775374,-70.99517583847046 44.3607763701854,-70.99680662155151 44.36571602660432,-70.99599123001099 44.368722570068854,-70.99448919296265 44.36961223191264,-70.99242925643921 44.37123813071097,-70.99054098129272 44.371422191881805,-70.98955392837524 44.372986688478704,-70.99075555801392 44.37507261892906,-70.98963975906372 44.37691308409485,-70.98848104476929 44.37832406821415,-70.9874939918518 44.38086988832067,-70.98470449447632 44.382403461463625,-70.98273038864136 44.384059675338044,-70.9810996055603 44.38531713976433,-70.97848176956177 44.3864825704798,-70.97749471664429 44.38746396782254,-70.97903966903687 44.38970271892836)',) db.session.add(wild_section) # Create a gage wild_gage = Gage(name='Wild River at Gilead', slug='wild-river-gilead', point='SRID=4326;POINT(-70.97963511943817 44.390833083196924)', river=wild, visible=True, zipcode='04217', local_town='Gilead, ME', location='Wild River at Gilead above Rt 2', key='password') db.session.add(wild_gage) # Create a sensor wild_sensor = Sensor(name='Gage Height', stype='usgs-height', local=False, remote_type='usgs', remote_id='01054200', gage=wild_gage) db.session.add(wild_sensor) # Create a sample wild_sample = Sample(sensor=wild_sensor, datetime=datetime.datetime.now(), value=5.8) db.session.add(wild_sample) db.session.commit()
def create(request): if request.method == 'POST': dataset = Dataset() dataset.owner = request.user dataset.name = request.POST['name'] dataset.number_of_labels = request.POST['number_of_labels'] dataset.description = request.POST['description'] if not dataset.privacy_validation(request.POST['privacy']): return render(request, '400.html', status=400) dataset_file = request.FILES['dataset'] reader = csv.reader(dataset_file, encoding='utf-8') header_list = reader.next() label_name = request.POST.get('label_name', 'CLASS') append_label_column = request.POST.get('append_label_column', False) if not append_label_column: label_index = header_list.index(label_name) header_list.pop(label_index) header_list.append(label_name) dataset.header = csvlist_to_string(header_list).strip() dataset.save() samples_count = 0 for row_list in reader: samples_count += 1 if not append_label_column: label_string = row_list.pop(label_index) row = csvlist_to_string(row_list).strip() sample = Sample(dataset=dataset, data=row, original_index=samples_count) sample.save() if not append_label_column and label_string: label = Label(owner=request.user, sample=sample, label=label_string) label.save() sample.times_labeled = 1 sample.save() dataset.number_of_samples = samples_count dataset.save() return HttpResponseRedirect( reverse('datasets_show', args=(dataset.id, )))
def sample(): """[summary] Returns: [type] -- [description] """ form = SampleForm() if form.validate_on_submit(): sample = Sample(id_sample=form.id_sample.data, num_seq=form.num_seq.data, date_time=form.date_time.data, organism=form.organism.data, location=form.location.data, batch=form.batch.data, path_r1=form.path_r1.data, path_r2=form.path_r2,result1=form.result1.data,result2=form.result2.data) #save the model to the database db.session.add(sample) db.session.commit() flash('Congratulations, you are now a registered sample!') return redirect(url_for('index')) return render_template('sample.html', title='validate', form=form)
def add_new_sample(sensor_id, dt, svalue, deltaminutes=10): """ Adds a new sample with an associacted remote sensor if there hasn't been a sample recorded within the last deltaminutes (default is 10). Arguments: sensor_id (int): Primary key for sensor dt (datetime): Datetime of sample svalue (float): Value of sample """ delta = datetime.datetime.now() - datetime.timedelta(minutes=deltaminutes) sample = Sample.query.filter_by(sensor_id=sensor_id)\ .order_by(Sample.datetime.desc()).first() if sample is None or (sample.datetime < delta and (sample.datetime != dt.replace(tzinfo=None))): new_sample = Sample(sensor_id=sensor_id, value=svalue, datetime=dt) db.session.add(new_sample) db.session.commit() logger.debug('Saved sample (%s - %s - %s) for sensor %s', new_sample.id, svalue, dt, sensor_id) else: message = '' if sample.datetime == dt.replace(tzinfo=None): message = message + 'Sample times are the same. ' if sample.datetime < delta: message = message + 'Last sample was more than {0} min ago '.format(deltaminutes) else: message = message + 'Last sample was less than {0} min ago '.format(deltaminutes) logger.warning('Discarded sample (%s - %s) for sensor %s, compared to (%s - %s). %s', svalue, dt, sensor_id, sample.value, sample.datetime, message)
def preprocess(sample): """Preprocess files after upload. :param sample: :class:`~app.models.Sample` :return: """ hash_path = os.path.join( current_app.config['APP_UPLOADS_SAMPLES'], sample.sha256 ) if zipfile.is_zipfile(hash_path): mt = magic.from_file(hash_path, mime=True) if mt in skip_mimes: return None current_app.log.debug('Extracting {}'.format(hash_path)) zfile = zipfile.ZipFile(hash_path) for zipfo in zfile.namelist(): cfg = current_app.config if zfile.getinfo(zipfo).compress_type == 99: # PK compat. v5.1 pwd = '-p{}'.format(cfg['INFECTED_PASSWD']) with popen('7z', 'e', '-so', pwd, hash_path) as zproc: buf, stderr = zproc.communicate() else: buf = zfile.read(zipfo, pwd=bytes(cfg['INFECTED_PASSWD'], 'utf-8')) digests = get_hashes(buf) hash_path = os.path.join(cfg['APP_UPLOADS_SAMPLES'], digests.sha256) if not os.path.isfile(hash_path): with open(hash_path, 'wb') as wf: wf.write(buf) s = Sample(user_id=sample.user_id, filename=zipfo, parent_id=sample.id, md5=digests.md5, sha1=digests.sha1, sha256=digests.sha256, sha512=digests.sha512, ctph=digests.ctph) db.session.add(s) db.session.commit()
def validate(id): order = Order.query.get(id) data_xls = pd.read_excel(order.file_url) for index, row in data_xls.iterrows(): # one patient have samples that have results # check patient number first if exist in db per row patient = Patient() patient_count = Patient.query.filter_by(code=row[0]).count() if patient_count > 0: sample_count = Sample.query.filter_by(code=row[7]).count() if sample_count > 0: # add new resltuts print(row[0] + ' ' + str(index)) else: sample = Sample() sample.origin_id = 1 sample.bio_code = patient.bio_code sample.volume = row[16] sample_nature = SampleNature.query.filter_by( name=row[8]).first() sample.sample_nature_id = sample_nature.id print(row[14]) sample_type = SampleType.query.filter_by(name=row[9]).first() sample.sample_type_id = sample_type.id support = Support.query.filter_by(name=row[13]).first() sample.support_id = support.id jonc_type = JoncType.query.filter_by(name=row[14]).first() if jonc_type is not None: sample.jonc_type_id = jonc_type.id mesure = Mesure.query.filter_by(name=row[17]).first() sample.mesure_id = mesure.id sample.code = row[7] sample.date = str(row[10]) sample.technique = row[11] sample.site = row[12] sample.status = 0 patient.samples.append(sample) order.patients.append(patient) db.session.commit() generateCode(sample, index + 1) else: # add new patient and samples and result of sample patient.code = row[0] patient.origin_id = 1 patient.bio_code = get_bio_code('HU') patient.age = row[1] patient.sexe = row[2] patient.birthday = str(row[3]) patient.city = row[4] patient.job = row[5] patient.clinical_data = row[6] # samples implementation for r in range(int(row[15])): sample = Sample() sample.origin_id = 1 sample.bio_code = patient.bio_code sample.volume = row[16] sample_nature = SampleNature.query.filter_by( name=row[8]).first() sample.sample_nature = sample_nature sample_type = SampleType.query.filter_by(name=row[9]).first() sample.sample_type = sample_type support = Support.query.filter_by(name=row[13]).first() sample.support = support jonc_type = JoncType.query.filter_by(name=row[14]).first() sample.jonc_type = jonc_type mesure = Mesure.query.filter_by(name=row[17]).first() sample.mesure_id = mesure.id sample.code = row[7] sample.date = str(row[10]) sample.technique = row[11] sample.site = row[12] sample.status = 0 sample.number = int(row[15]) patient.samples.append(sample) generateCode(sample, r + 1) print(str(sample.bio_code)) order.patients.append(patient) db.session.commit() order.status = 1 db.session.commit() # and check sample id if already in db flash(_('Traitement effectué avec succes!')) return redirect(url_for('order.detail', id=order.id))
def do_import(id): order = Order.query.get(id) if request.method == 'POST': file = request.files['file'] filename = documents.save(file) data_xls = pd.read_excel(file) for index, row in data_xls.iterrows(): # one patient have samples that have results # check patient number first if exist in db per row patient = Patient() patient_count = Patient.query.filter_by(code=row[0]).count() if patient_count > 0: sample_count = Sample.query.filter_by(code=row[7]).count() if sample_count > 0: # add new resltuts print(row[0] + ' ' + str(index)) else: sample = Sample() sample.bio_code = patient.bio_code sample.volume = row[16] sample_nature = SampleNature.query.filter_by( name=row[8]).first() sample.sample_nature_id = sample_nature.id sample_type = SampleType.query.filter_by( name=row[9]).first() sample.sample_type_id = sample_type.id support = Support.query.filter_by(name=row[13]).first() sample.support_id = support.id jonc_type = JoncType.query.filter_by(name=row[14]).first() sample.jonc_type_id = jonc_type.id mesure = Mesure.query.filter_by(name=row[17]).first() sample.mesure_id = mesure.id sample.code = row[7] sample.date = str(row[10]) sample.technique = row[11] sample.site = row[12] sample.status = 0 patient.samples.append(sample) order.patients.append(patient) db.session.commit() generateCode(sample, r + 1) else: # add new patient and samples and result of sample print(row[4] + ' ' + str(index)) patient.code = row[0] patient.origin_id = 1 patient.bio_code = get_bio_code('HU') patient.age = row[1] patient.sexe = row[2] patient.birthday = str(row[3]) patient.city = row[4] patient.job = row[5] patient.clinical_data = row[6] # samples implementation for r in range(int(row[15])): sample = Sample() sample.bio_code = patient.bio_code sample.volume = row[16] sample_nature = SampleNature.query.filter_by( name=row[8]).first() sample.sample_nature_id = sample_nature.id sample_type = SampleType.query.filter_by( name=row[9]).first() sample.sample_type_id = sample_type.id support = Support.query.filter_by(name=row[13]).first() sample.support_id = support.id jonc_type = JoncType.query.filter_by(name=row[14]).first() sample.jonc_type_id = jonc_type.id mesure = Mesure.query.filter_by(name=row[17]).first() sample.mesure_id = mesure.id sample.code = row[7] sample.date = str(row[10]) sample.technique = row[11] sample.site = row[12] sample.status = 0 patient.samples.append(sample) generateCode(sample, r + 1) print(str(sample.bio_code)) order.patients.append(patient) db.session.commit() # and check sample id if already in db url = documents.url(filename) document = Document(name=filename, url=url) order.documents.append(document) db.session.add(document) db.session.commit() flash(_('Fichier importé avec succes!')) return redirect(url_for('order.detail', id=order.id)) return render_template('order/import.html', order=order)
def _create_url_sample(url): sha256 = hashlib.sha256(url.encode('utf-8')).hexdigest() return Sample(filename=url, sha256=sha256, user_id=g.user.id, md5='N/A', sha1='N/A', sha512='N/A', ctph='N/A')
def setUp(self): self.app = create_app('testing') self.client = self.app.test_client() self.app_context = self.app.app_context() self.app_context.push() db.create_all() # Create some data to test against, but don't commit # Create a region maine = Region(name='Maine', slug='maine') db.session.add(maine) # Create a river andro = River(name='Androscoggin River', slug='androscoggin') wild = River(name='Wild River of the Androsoggin', slug='wild-river', parent=andro) db.session.add(andro) db.session.add(wild) # Create a section wild_section = Section( name='Wild River from Hastings', slug='hasting-to-gilead', river=wild, putin='SRID=4326;POINT(-71.05996191501617 44.31595096222731)', takeout='SRID=4326;POINT(-70.97963511943817 44.390833083196924)', path= 'SRID=4326;LINESTRING(-71.05997800827026 44.316024368364864,-71.05881929397583 44.31798950769032,-71.05731725692749 44.31884923545022,-71.05444192886353 44.31851148676115,-71.05298280715942 44.31943261497028,-71.05096578598022 44.322687152160796,-71.05045080184937 44.32449856163325,-71.04984998703003 44.32495908054771,-71.04761838912964 44.325849406864485,-71.04568719863892 44.32649411748597,-71.04306936264038 44.32753791965937,-71.04049444198608 44.327138821021585,-71.03847742080688 44.32664761897048,-71.03607416152954 44.32572660403795,-71.03517293930054 44.32554239931617,-71.03431463241577 44.32594150881567,-71.03341341018677 44.32805981378074,-71.03264093399048 44.329103588092785,-71.02929353713989 44.32984035877702,-71.02843523025513 44.33008594694842,-71.02757692337036 44.33137526797721,-71.02804899215698 44.33309431859246,-71.02783441543579 44.33459844654652,-71.02543115615845 44.33582627750024,-71.02311372756958 44.33714616710255,-71.0210108757019 44.33926406740166,-71.01598978042603 44.34328479806066,-71.01294279098511 44.3441441551062,-71.00916624069214 44.346752840399844,-71.0082221031189 44.34972966884455,-71.00689172744751 44.35107993293547,-71.00551843643188 44.351171995263435,-71.00393056869507 44.350374116950185,-71.00148439407349 44.35000586175751,-70.9984803199768 44.350374116950185,-70.99642038345337 44.35163230473401,-70.99328756332397 44.354117913402796,-70.9923005104065 44.3552225945275,-70.99212884902954 44.35678752380721,-70.99401712417603 44.357830786775374,-70.99517583847046 44.3607763701854,-70.99680662155151 44.36571602660432,-70.99599123001099 44.368722570068854,-70.99448919296265 44.36961223191264,-70.99242925643921 44.37123813071097,-70.99054098129272 44.371422191881805,-70.98955392837524 44.372986688478704,-70.99075555801392 44.37507261892906,-70.98963975906372 44.37691308409485,-70.98848104476929 44.37832406821415,-70.9874939918518 44.38086988832067,-70.98470449447632 44.382403461463625,-70.98273038864136 44.384059675338044,-70.9810996055603 44.38531713976433,-70.97848176956177 44.3864825704798,-70.97749471664429 44.38746396782254,-70.97903966903687 44.38970271892836)', ) db.session.add(wild_section) # Create a gage wild_gage = Gage( name='Wild River at Gilead', slug='wild-river-gilead', point='SRID=4326;POINT(-70.97963511943817 44.390833083196924)', river=wild, visible=True, zipcode='04217', local_town='Gilead, ME', location='Wild River at Gilead above Rt 2', key='password') db.session.add(wild_gage) # Create a sensor wild_sensor = Sensor(name='Gage Height', stype='usgs-height', local=False, remote_type='usgs', remote_id='01054200', gage=wild_gage) diamond_height = Sensor(name='Gage Height', stype='usgs-height', local=False, remote_type='usgs', remote_id='01052500') diamond_discharge = Sensor(name='Discharge', stype='usgs-discharge', local=False, remote_type='usgs', remote_id='01052500', remote_parameter='00060') rapid_cfs = Sensor(name='Rapid CFS', stype='h2oline-cfs', local=False, remote_type='h2oline', remote_id='235127') azicohos_level = Sensor(name='Aziscohol height', stype='h2oline-height', local=False, remote_type='h2oline', remote_id='235130', remote_parameter='FT') neilson_flow = Sensor(name='Neilson flow', stype='cehq-flow', local=False, remote_type='cehq', remote_id='050915') skeena_level = Sensor(name='Skeena Level', stype='cawater-level', local=False, remote_type='cawater', remote_id='BC_08EB003') humber_level = Sensor(name='Humber Level', stype='cawater-level', local=False, remote_type='cawater', remote_id='NL_02YL012') cheakamus_discharge = Sensor(name='Cheakamus Discharge', stype='cawater-discharge', local=False, remote_type='cawater', remote_id='BC_08GA043', remote_parameter='discharge') canaseraga_stage = Sensor(name='Canaseraga Creek', stype='canaseraga-stage', local=False, remote_type='corps', remote_id='DSVN6') db.session.add(wild_sensor) db.session.add(diamond_height) db.session.add(diamond_discharge) db.session.add(rapid_cfs) db.session.add(azicohos_level) db.session.add(neilson_flow) db.session.add(skeena_level) db.session.add(humber_level) db.session.add(canaseraga_stage) # Create a sample dt = datetime.datetime.now() - datetime.timedelta(hours=12) wild_sample = Sample(sensor=wild_sensor, datetime=dt, value=5.8) db.session.add(wild_sample) wild_correlation = Correlation(sensor=wild_sensor, section=wild_section, minimum=3.5, low=4.0, medium=5.0, high=6.0, huge=9.0) db.session.add(wild_correlation) db.session.commit()
def add_sample(): """Upload untrusted files, E.i. malware samples, files for analysis. Uploaded files are save in :attr:`config.Config.APP_UPLOADS_SAMPLES` using the SHA-256 of the content as file name. Existing files are not overwritten. After upload MD5, SHA1, SHA256, SHA512 and CTPH hashes are calculated. **Example request**: .. sourcecode:: http POST /api/1.0/samples HTTP/1.1 Host: do.cert.europa.eu Accept: application/json Content-Length: 317825 Content-Type: multipart/form-data; boundary=-----------3fa8efc8eb2a42e7 Content-Disposition: form-data; name="files[0]"; filename="zepto.exe" **Example response**: .. sourcecode:: http HTTP/1.0 201 CREATED Content-Type: application/json { "files": [ { "created": "2016-08-02T14:26:15", "ctph": "6144:dOWrXkMZWMKsLXiyLgDf1tedfmqmqeGGAV//CNGa1FPi:d3rV", "filename": "zepto.exe", "id": 32, "md5": "a8188e964bc1f9cb1e905ce8f309e086", "sha1": "c3db12e0ffc4b4b090e32679c95aaa76e07150f7", "sha256": "7768d4e54b066a567bed1456077025ba7eb56a88aed1bc8cb207", "sha512": "1fa1ea4a72be8adc9257185a9d71d889fbea2360cee3f6102302e" } ], "message": "Files uploaded" } :reqheader Accept: Content type(s) accepted by the client :reqheader Content-Type: multipart/form-data required :resheader Content-Type: this depends on `Accept` header or request :form files: Files to be uploaded :>json array files: List of files saved to disk :>jsonarr integer id: Sample unique ID :>jsonarr string created: Time of upload :>jsonarr string sha256: SHA256 of file :>jsonarr string ctph: CTPH (a.k.a. fuzzy hash) of file :>jsonarr string filename: Filename (as provided by the client) :>json string message: Status message :statuscode 201: Files successfully saved """ uploaded_samples = [] for idx, file in request.files.items(): buf = file.stream.read() digests = get_hashes(buf) hash_path = os.path.join(current_app.config['APP_UPLOADS_SAMPLES'], digests.sha256) if not os.path.isfile(hash_path): file.stream.seek(0) file.save(hash_path) s = Sample(user_id=g.user.id, filename=file.filename, md5=digests.md5, sha1=digests.sha1, sha256=digests.sha256, sha512=digests.sha512, ctph=digests.ctph) db.session.add(s) try: db.session.commit() analysis.preprocess(s) except Exception as e: db.session.rollback() db.session.flush() current_app.log.error(e.args[0]) uploaded_samples.append(s.serialize()) return ApiResponse({ 'message': 'Files uploaded', 'files': uploaded_samples }, 201)
def add_fireeye_url_analysis(): """Submit URLs to the FireEye Sandbox. Also accepts :http:method:`put`. .. warning:: Not Implemented **Example request**: .. sourcecode:: http POST /api/1.0/analysis/fireeye-url HTTP/1.1 Host: do.cert.europa.eu Accept: application/json Content-Type: application/json { "urls": ["http://cert.europa.eu"], "dyn_analysis": { "fireeye": [5, 6] } } **Example response**: .. sourcecode:: http HTTP/1.0 200 OK Content-Type: application/json { "message": "Your URLs have been submitted for dynamic analysis", "statuses": [ { "sha256": "33a53e3b28ee41c29afe79f49ecc53b34ac125e5e15f9e7c..." } ] } :reqheader Accept: Content type(s) accepted by the client :resheader Content-Type: this depends on `Accept` header or request :<json array files: List of URLs to scan :<jsonarr string sha256: SHA256 of the shortcut file created :<json object dyn_analysis: Sandbox configuration :<jsonarr array fireeye: List of FireEye environments to submit to Get the list from :http:get:`/api/1.0/analysis/fireeye/environments` :>json array statuses: List of statuses returned by upstream APIs :>jsonarr string error: Error message from upstream API :>jsonarr string sha256: SHA256 calculated by upstream API :>json string message: Status message :status 202: The URLs have been accepted for scanning :status 400: Bad request """ raise ApiException({}, 501) statuses = [] samples = {} for env in request.json['dyn_analysis']['fireeye']: for url in request.json['urls']: sdata = {'environmentId': env, 'analyzeurl': url} headers = { 'User-Agent': 'FireEye Sandbox API Client', 'Accept': 'application/json' } resp = fireeye.submiturl(sdata, headers=headers) samples[resp['response']['sha256']] = url statuses.append(resp['response']) if resp['response_code'] != 0: current_app.log.debug(resp) for sha256, url in samples.items(): surl = Sample(filename=url, sha256=sha256, user_id=g.user.id, md5='N/A', sha1='N/A', sha512='N/A', ctph='N/A') db.session.add(surl) db.session.commit() return { 'statuses': statuses, 'message': 'Your URLs have been submitted for dynamic analysis' }, 202
def add_cp_sample(): """Upload untrusted files, E.i. malware samples, files for analysis. After upload MD5, SHA1, SHA256, SHA512 and CTPH hashes are calculated. **Example request**: .. sourcecode:: http POST /api/1.0/samples HTTP/1.1 Host: cp.cert.europa.eu Accept: application/json Content-Type: multipart/form-data; boundary=----FormBoundaryrflTTZA0oE ------FormBoundaryrflTTZA0oE Content-Disposition: form-data; name="files[0]"; filename="stux.zip" Content-Type: application/zip ------FormBoundaryrflTTZA0oE-- **Example response**: .. sourcecode:: http HTTP/1.0 201 CREATED Content-Type: application/json { "files": [ { "created": "2016-03-21T16:09:47", "ctph": "49152:77qzLl6EKvwkdB7qzLl6EKvwkTY40GfAHw7qzLl6EKvwk...", "filename": "stux.zip", "id": 2, "sha256": "1eedab2b09a4bf6c87b273305c096fa2f597ff9e4bdd39bc4..." } ], "message": "Files uploaded" } :reqheader Accept: Content type(s) accepted by the client :reqheader Content-Type: multipart/form-data required :resheader Content-Type: this depends on `Accept` header or request :form files: Files to be uploaded :>json array files: List of files saved to disk :>jsonarr integer id: Sample unique ID :>jsonarr string created: Time of upload :>jsonarr string sha256: SHA256 of file :>jsonarr string ctph: CTPH (a.k.a. fuzzy hash) of file :>jsonarr string filename: Filename (as provided by the client) :>json string message: Status message :statuscode 201: Files successfully saved """ uploaded_samples = [] for idx, file_ in request.files.items(): buf = file_.stream.read() hashes = get_hashes(buf) hash_path = os.path.join(current_app.config['APP_UPLOADS_SAMPLES'], hashes.sha256) if not os.path.isfile(hash_path): file_.stream.seek(0) file_.save(hash_path) s = Sample(user_id=g.user.id, filename=file_.filename, md5=hashes.md5, sha1=hashes.sha1, sha256=hashes.sha256, sha512=hashes.sha512, ctph=hashes.ctph) db.session.add(s) try: db.session.commit() analysis.preprocess(s) except Exception as e: db.session.rollback() db.session.flush() current_app.log.error(e.args[0]) uploaded_samples.append(s.serialize()) return ApiResponse({ 'message': 'Files uploaded', 'files': uploaded_samples }, 201)