On left there is an array of dimension name that will generate the
dimension vales on the right
Left:Right
['company', 'company_id', 'company_code']: ['ABC', 'ACME', 'JDI']
['plant', 'plant_id', 'plant_code']: ['Zhongshun', 'Holtsburg', 'Midway']
['plant', 'plant_id', 'plant_code']: ['US', 'CA', 'UK', 'DE']
['firmware', 'firmware_version']: ['1.0', '1.12', '1.13', '2.1']
['manufacturer']: ['Rentech', 'GHI Industries']
['zone']: ['27A', '27B', '27C']
['status', 'status_code']: ['inactive', 'active']
['operator', 'operator_id', 'person', 'employee']: ['Fred', 'Joe', 'Mary', 'Steve', 'Henry', 'Jane', 'Hillary', 'Justin', 'Rod']
3.3 Other dimension name
Any other dimension name other than the one above will generate random values
'''
db.drop_table(dim_table_name, schema=schema)
entity_type.make_dimension(dim_table_name, Column('company', String(50)),
Column('status', String(50)),
Column('operator', String(50)),
**{'schema': schema})
entity_type.register()
'''
To test the execution of kpi calculations defined for the entity type locally
use this function.
A local test will not update the server job log or write kpi data to the AS data
lake. Instead kpi data is written to the local filesystem in csv form.
'''
# entity_type.exec_local_pipeline(**{'_production_mode': False})
'''
Create a database object to access Watson IOT Platform Analytics DB.
'''
db = Database(credentials = credentials)
db_schema = None # set if you are not using the default
'''
To do anything with IoT Platform Analytics, you will need one or more entity type.
You can create entity types through the IoT Platform or using the python API as shown below.
The database schema is only needed if you are not using the default schema. You can also rename the timestamp.
'''
entity_name = settings.ENTITY_NAME or 'buildings'
db_schema = settings.DB_SCHEMA or "BLUADMIN" # None # replace if you are not using the default schema
db.drop_table(entity_name, schema = db_schema)
# Credentials to access Building Insights API.
USERNAME = settings.USERNAME
PASSWORD = settings.PASSWORD
# TENANT_ID = settings.TENANT_ID
URL = settings.URL
entity = EntityType(entity_name, db,
# following columns can be dynamically generated based on meters associated with each asset
Column('building',String(50)),
# Column('energy_value',Float()),
# Column('energy_unit',String(50)),
Column('temperature',Float()),
Column('temperature_unit',String(50)),
'_db_schema': db_schema
})
'''
Now that we have defined simulation parameters and attached them to an EntityDataGenerater
object on an EntityType we can run the AS calculation pipeline to execute the generator.
When running the calculation pipeline it is important to set a start_date in the past. Since
these dimensions change slowly, we will only see changes when looking a large enough
time window.
The default frequency for automatic generation of SCD data is 2 days. By starting the pipeline
30 days in the past we will see history.
'''
db.drop_table(scd_name, schema=db_schema)
start_date = dt.datetime.utcnow() - dt.timedelta(days=30)
entity.exec_local_pipeline(start_ts=start_date)
'''
After running this, you can take a look at the structure and content of the SCD input
table that was generated. When building your own SCD table and populating it manually
you will need to conform to this structure.
deviceid start_date end_date owner
73004 2019-07-24-22.10.25.940354 2019-07-27-22.10.25.940353 Jane S
73004 2019-07-27-22.10.25.940354 2019-08-14-22.10.25.940353 Steve S
73004 2019-08-14-22.10.25.940354 2019-08-15-22.10.25.940353 Fred K
73004 2019-08-15-22.10.25.940354 2019-08-20-22.10.25.940353 Jane S
73004 2019-08-20-22.10.25.940354 2019-08-21-22.10.25.940353 John H
73004 2019-08-21-22.10.25.940354 2262-04-11-23.47.16.854775 Harry L
'''
Create a database object to access Watson IOT Platform Analytics DB.
'''
db = Database(credentials=credentials)
db_schema = None
entity_name = 'Equipment' # choose a valid entity type name
test_entity = ServerEntityType(entity_name, db=db, db_schema=db_schema)
print(test_entity)
test_entity.exec_local_pipeline(_abort_on_fail=True)
'''
To do anything with IoT Platform Analytics, you will need one or more entity type.
You can create entity types through the IoT Platform or using the python API as shown below.
The database schema is only needed if you are not using the default schema. You can also rename the timestamp.
'''
entity_name = 'Equipment2'
'''
db.drop_table(entity_name, schema = db_schema)
entity = EntityType(entity_name,db,
Column('DEVICEID',String(50)),
Column('ASSET_ID', String(50)),
Column('ENTITY_ID', String(50)),
Column('drvn_t1',Float()),
Column('drvn_p1',Float()),
Column('STEP',Float()),
Column('PRESS_X',Float()),
Column('PRESS_Y',Float()),
Column('TEMP_X',Float()),
Column('TEMP_Y',Float()),
Column('TEMP_Y', Float()),
domain of values.
Let's add additional numeric and non-numeric data to the dimension and set
simulation parameters for them.
load_rating is a numeric dimension property with a mean of 500 and a default standard deviation
maintenance_org has a custom domain
Important: When altering the dimension it is important to drop it first. The make_dimension
method attempts to reuse an existing table if there is one. In this case there is one and it
contains the wrong columns. We must drop it first so that make_dimension can create
a new dimension.
'''
db.drop_table('sim_test_dimension', schema=db_schema)
entity.make_dimension('sim_test_dimension', Column('manufacturer', String(50)), Column('load_rating', Float()),
Column('maintenance_org', String(50)))
sim_parameters = {"data_item_mean": {'temp': 22, 'pressure': 320, 'load_rating': 500},
"data_item_sd": {'temp': 2, 'pressure': 5},
"data_item_domain": {'category_code': ['A', 'B', 'C'], 'maintenance_org': ['Lunar Parts', 'Relco']},
"start_entity_id": 1000, "auto_entity_count": 10, "freq": '30S' # 30 sec
}
entity.register(raise_error=True)
entity.generate_data(days=0.5, drop_existing=True, **sim_parameters)
'''
x0, x1 and x2 are independent random variables
y0 is also an independent random variable - it will not be possible to predict this variable
y1 is a direct linear function of x1 and x2 - it will be a breeze to predict
y2 is y1 with some added noise thrown in to make it more difficult to fit a model
y3 has a non linear relation to x1 and x2, but will be easy to predict with the right estimator
y4 is y3 with some noise
We will start by trying to predict the easy on: y1 using the SimpleRegressor function.
'''
entity_name = 'predict_test' # you can give your entity type a better nane
db = Database(credentials=credentials)
db_schema = None # set if you are not using the default
db.drop_table(entity_name)
fn_gen = bif.EntityDataGenerator(output_item='generator_ok')
fn_dep1 = bif.PythonExpression( # linear relatoionship
'5*df["x1"]-df["x2"]', 'y1')
fn_dep2 = bif.PythonExpression(
'df["x1"]*df["x1"]-df["x2"]', # non-linear relationship
'y3')
fn_noise = bif.RandomNoise( # add noise to y1 and y3 to produce y2 and y4
input_items=['y1', 'y3'],
standard_deviation=.5,
output_items=['y2', 'y4'])
job_settings = {
'delete_existing_models': True,
}
'''
'''
1. Create a database object to access Watson IOT Platform Analytics DB.
'''
db = Database(credentials=credentials)
'''
2. To do anything with IoT Platform Analytics, you will need one or more entity type.
You can create entity types through the IoT Platform or using the python API as shown below.
When creating a new entity type you can create it's corresponding dimension table as shown below.
The database schema is only needed if you are not using the default schema. You can also rename the timestamp.
'''
entity_name = 'issue455_0'
entity_dimension_lower = entity_name.lower() + '_dimension'
entity_dimension_upper = entity_name.upper() + '_dimension'
db.drop_table(entity_name, schema=db_schema)
db.drop_table(entity_dimension_lower, schema=db_schema)
db.drop_table(entity_dimension_upper, schema=db_schema)
entity = EntityType(
entity_name, db, Column('TURBINE_ID', String(50)),
Column('TEMPERATURE', Float()), Column('PRESSURE', Float()),
Column('STEP', Float()), Column('PRESS_X', Float()),
Column('PRESS_Y', Float()), Column('TEMP_X', Float()),
Column('TEMP_Y', Float()),
Issue455HTTPPreload(
request='GET',
url="https://turbine-simulator.mybluemix.net/v1/api/reading",
output_item='http_preload_done'), **{
'_timestamp': 'evt_timestamp',
'_db_schema': db_schema
