On left there is an array of dimension name that will generate the dimension vales on the right Left:Right ['company', 'company_id', 'company_code']: ['ABC', 'ACME', 'JDI'] ['plant', 'plant_id', 'plant_code']: ['Zhongshun', 'Holtsburg', 'Midway'] ['plant', 'plant_id', 'plant_code']: ['US', 'CA', 'UK', 'DE'] ['firmware', 'firmware_version']: ['1.0', '1.12', '1.13', '2.1'] ['manufacturer']: ['Rentech', 'GHI Industries'] ['zone']: ['27A', '27B', '27C'] ['status', 'status_code']: ['inactive', 'active'] ['operator', 'operator_id', 'person', 'employee']: ['Fred', 'Joe', 'Mary', 'Steve', 'Henry', 'Jane', 'Hillary', 'Justin', 'Rod'] 3.3 Other dimension name Any other dimension name other than the one above will generate random values ''' db.drop_table(dim_table_name, schema=schema) entity_type.make_dimension(dim_table_name, Column('company', String(50)), Column('status', String(50)), Column('operator', String(50)), **{'schema': schema}) entity_type.register() ''' To test the execution of kpi calculations defined for the entity type locally use this function. A local test will not update the server job log or write kpi data to the AS data lake. Instead kpi data is written to the local filesystem in csv form. ''' # entity_type.exec_local_pipeline(**{'_production_mode': False})
''' Create a database object to access Watson IOT Platform Analytics DB. ''' db = Database(credentials = credentials) db_schema = None # set if you are not using the default ''' To do anything with IoT Platform Analytics, you will need one or more entity type. You can create entity types through the IoT Platform or using the python API as shown below. The database schema is only needed if you are not using the default schema. You can also rename the timestamp. ''' entity_name = settings.ENTITY_NAME or 'buildings' db_schema = settings.DB_SCHEMA or "BLUADMIN" # None # replace if you are not using the default schema db.drop_table(entity_name, schema = db_schema) # Credentials to access Building Insights API. USERNAME = settings.USERNAME PASSWORD = settings.PASSWORD # TENANT_ID = settings.TENANT_ID URL = settings.URL entity = EntityType(entity_name, db, # following columns can be dynamically generated based on meters associated with each asset Column('building',String(50)), # Column('energy_value',Float()), # Column('energy_unit',String(50)), Column('temperature',Float()), Column('temperature_unit',String(50)),
'_db_schema': db_schema }) ''' Now that we have defined simulation parameters and attached them to an EntityDataGenerater object on an EntityType we can run the AS calculation pipeline to execute the generator. When running the calculation pipeline it is important to set a start_date in the past. Since these dimensions change slowly, we will only see changes when looking a large enough time window. The default frequency for automatic generation of SCD data is 2 days. By starting the pipeline 30 days in the past we will see history. ''' db.drop_table(scd_name, schema=db_schema) start_date = dt.datetime.utcnow() - dt.timedelta(days=30) entity.exec_local_pipeline(start_ts=start_date) ''' After running this, you can take a look at the structure and content of the SCD input table that was generated. When building your own SCD table and populating it manually you will need to conform to this structure. deviceid start_date end_date owner 73004 2019-07-24-22.10.25.940354 2019-07-27-22.10.25.940353 Jane S 73004 2019-07-27-22.10.25.940354 2019-08-14-22.10.25.940353 Steve S 73004 2019-08-14-22.10.25.940354 2019-08-15-22.10.25.940353 Fred K 73004 2019-08-15-22.10.25.940354 2019-08-20-22.10.25.940353 Jane S 73004 2019-08-20-22.10.25.940354 2019-08-21-22.10.25.940353 John H 73004 2019-08-21-22.10.25.940354 2262-04-11-23.47.16.854775 Harry L
''' Create a database object to access Watson IOT Platform Analytics DB. ''' db = Database(credentials=credentials) db_schema = None entity_name = 'Equipment' # choose a valid entity type name test_entity = ServerEntityType(entity_name, db=db, db_schema=db_schema) print(test_entity) test_entity.exec_local_pipeline(_abort_on_fail=True) ''' To do anything with IoT Platform Analytics, you will need one or more entity type. You can create entity types through the IoT Platform or using the python API as shown below. The database schema is only needed if you are not using the default schema. You can also rename the timestamp. ''' entity_name = 'Equipment2' ''' db.drop_table(entity_name, schema = db_schema) entity = EntityType(entity_name,db, Column('DEVICEID',String(50)), Column('ASSET_ID', String(50)), Column('ENTITY_ID', String(50)), Column('drvn_t1',Float()), Column('drvn_p1',Float()), Column('STEP',Float()), Column('PRESS_X',Float()), Column('PRESS_Y',Float()), Column('TEMP_X',Float()), Column('TEMP_Y',Float()), Column('TEMP_Y', Float()),
domain of values. Let's add additional numeric and non-numeric data to the dimension and set simulation parameters for them. load_rating is a numeric dimension property with a mean of 500 and a default standard deviation maintenance_org has a custom domain Important: When altering the dimension it is important to drop it first. The make_dimension method attempts to reuse an existing table if there is one. In this case there is one and it contains the wrong columns. We must drop it first so that make_dimension can create a new dimension. ''' db.drop_table('sim_test_dimension', schema=db_schema) entity.make_dimension('sim_test_dimension', Column('manufacturer', String(50)), Column('load_rating', Float()), Column('maintenance_org', String(50))) sim_parameters = {"data_item_mean": {'temp': 22, 'pressure': 320, 'load_rating': 500}, "data_item_sd": {'temp': 2, 'pressure': 5}, "data_item_domain": {'category_code': ['A', 'B', 'C'], 'maintenance_org': ['Lunar Parts', 'Relco']}, "start_entity_id": 1000, "auto_entity_count": 10, "freq": '30S' # 30 sec } entity.register(raise_error=True) entity.generate_data(days=0.5, drop_existing=True, **sim_parameters) '''
x0, x1 and x2 are independent random variables y0 is also an independent random variable - it will not be possible to predict this variable y1 is a direct linear function of x1 and x2 - it will be a breeze to predict y2 is y1 with some added noise thrown in to make it more difficult to fit a model y3 has a non linear relation to x1 and x2, but will be easy to predict with the right estimator y4 is y3 with some noise We will start by trying to predict the easy on: y1 using the SimpleRegressor function. ''' entity_name = 'predict_test' # you can give your entity type a better nane db = Database(credentials=credentials) db_schema = None # set if you are not using the default db.drop_table(entity_name) fn_gen = bif.EntityDataGenerator(output_item='generator_ok') fn_dep1 = bif.PythonExpression( # linear relatoionship '5*df["x1"]-df["x2"]', 'y1') fn_dep2 = bif.PythonExpression( 'df["x1"]*df["x1"]-df["x2"]', # non-linear relationship 'y3') fn_noise = bif.RandomNoise( # add noise to y1 and y3 to produce y2 and y4 input_items=['y1', 'y3'], standard_deviation=.5, output_items=['y2', 'y4']) job_settings = { 'delete_existing_models': True, }
''' ''' 1. Create a database object to access Watson IOT Platform Analytics DB. ''' db = Database(credentials=credentials) ''' 2. To do anything with IoT Platform Analytics, you will need one or more entity type. You can create entity types through the IoT Platform or using the python API as shown below. When creating a new entity type you can create it's corresponding dimension table as shown below. The database schema is only needed if you are not using the default schema. You can also rename the timestamp. ''' entity_name = 'issue455_0' entity_dimension_lower = entity_name.lower() + '_dimension' entity_dimension_upper = entity_name.upper() + '_dimension' db.drop_table(entity_name, schema=db_schema) db.drop_table(entity_dimension_lower, schema=db_schema) db.drop_table(entity_dimension_upper, schema=db_schema) entity = EntityType( entity_name, db, Column('TURBINE_ID', String(50)), Column('TEMPERATURE', Float()), Column('PRESSURE', Float()), Column('STEP', Float()), Column('PRESS_X', Float()), Column('PRESS_Y', Float()), Column('TEMP_X', Float()), Column('TEMP_Y', Float()), Issue455HTTPPreload( request='GET', url="https://turbine-simulator.mybluemix.net/v1/api/reading", output_item='http_preload_done'), **{ '_timestamp': 'evt_timestamp', '_db_schema': db_schema