def main(): """Go Main""" total = None years = 0. for yr in range(1981, 2018): print(yr) ncfn = "/mesonet/data/prism/%s_daily.nc" % (yr, ) nc = netCDF4.Dataset(ncfn) if total is None: lons = nc.variables['lon'][:] lats = nc.variables['lat'][:] total = np.zeros(nc.variables['tmax'].shape[1:], np.float) days = np.zeros(nc.variables['tmax'].shape[1:], np.float) sidx = daily_offset(datetime.date(yr, 1, 1)) eidx = daily_offset(datetime.date(yr, 7, 4)) for idx in range(sidx, eidx): days += np.where(nc.variables['tmax'][idx, :, :] > THRESHOLD, 1, 0) nc.close() years += 1. total += days val = days - (total / years) print(np.max(val)) print(np.min(val)) mp = MapPlot(sector='conus', title=("OSU PRISM 2017 Days with High >= 90$^\circ$F " "Departure"), subtitle=("2017 thru 4 July against 1981-2016 " "Year to Date Average")) mp.contourf(lons, lats, val, np.arange(-25, 26, 5), units='days', cmap=plt.get_cmap('seismic')) mp.postprocess(filename='test.png')
def main(): """Go Main""" nt = NetworkTable(['AWOS', 'IA_ASOS']) pgconn = psycopg2.connect(database='mos', host='localhost', user='******', port=5555) df = read_sql(""" select station, avg(pwater) from model_gridpoint where model = 'NAM' and extract(hour from runtime at time zone 'UTC') = 0 and pwater > 0 and pwater < 100 and extract(month from runtime) between 4 and 9 and ftime = runtime GROUP by station ORDER by avg """, pgconn, index_col='station') df['lat'] = None df['lon'] = None df['pwater'] = distance(df['avg'].values, 'MM').value('IN') for station in df.index.values: df.at[station, 'lat'] = nt.sts[station[1:]]['lat'] df.at[station, 'lon'] = nt.sts[station[1:]]['lon'] mp = MapPlot(title=('00z Analysis NAM Warm-Season Average ' 'Precipitable Water [in]'), subtitle=("based on grid point samples " "from 2004-2017 (April-September)")) cmap = plt.get_cmap("plasma_r") cmap.set_under('white') cmap.set_over('black') mp.contourf(df['lon'], df['lat'], df['pwater'], np.arange(0.94, 1.13, 0.03), cmap=cmap, units='inch') mp.drawcounties() mp.drawcities() mp.postprocess(filename='170901.png') mp.close()
def plot_maxmin(ts, field): """Generate our plot.""" nc = ncopen(ts.strftime("/mesonet/data/ndfd/%Y%m%d%H_ndfd.nc")) if field == 'high_tmpk': data = np.max(nc.variables[field][:], 0) elif field == 'low_tmpk': data = np.min(nc.variables[field][:], 0) data = masked_array(data, units.degK).to(units.degF).m subtitle = ("Based on National Digital Forecast Database (NDFD) " "00 UTC Forecast made %s") % (ts.strftime("%-d %b %Y"), ) mp = MapPlot(title='NWS NDFD 7 Day (%s through %s) %s Temperature' % ( ts.strftime("%-d %b"), (ts + datetime.timedelta(days=6)).strftime("%-d %b"), 'Maximum' if field == 'high_tmpk' else 'Minimum', ), subtitle=subtitle, sector='iailin') mp.pcolormesh(nc.variables['lon'][:], nc.variables['lat'][:], data, np.arange(10, 121, 10), cmap=plt.get_cmap('jet'), units='Degrees F') mp.drawcounties() pqstr = ( "data c %s summary/cb_ndfd_7day_%s.png summary/cb_ndfd_7day_%s.png " "png") % (ts.strftime("%Y%m%d%H%M"), "max" if field == 'high_tmpk' else 'min', "max" if field == 'high_tmpk' else 'min') mp.postprocess(pqstr=pqstr) mp.close() nc.close()
def plot(argv): """Make a plot""" df = pd.read_csv("%s_maxdailyprecip.txt" % (argv[1], ), dtype={'huc12': str}) df.set_index('huc12', inplace=True) pgconn = get_dbconn('idep') huc12df = gpd.GeoDataFrame.from_postgis(""" SELECT huc12, simple_geom as geo from wbd_huc12 WHERE swat_use ORDER by huc12 """, pgconn, index_col='huc12', geom_col='geo') mp = MapPlot( sector='custom', south=34, north=48, west=-98, east=-77, title="%s Max Daily Precipitation" % (argv[1].split("_", 1)[1], ) ) bins = range(0, 201, 20) cmap = stretch_cmap('terrain_r', bins) norm = mpcolors.BoundaryNorm(bins, cmap.N) for huc12, row in huc12df.iterrows(): for poly in row['geo']: arr = np.asarray(poly.exterior) points = mp.ax.projection.transform_points(ccrs.Geodetic(), arr[:, 0], arr[:, 1]) color = cmap(norm([df.at[huc12, 'maxp'], ]))[0] poly = Polygon(points[:, :2], fc=color, ec='None', zorder=2, lw=.1) mp.ax.add_patch(poly) mp.draw_colorbar(bins, cmap, norm, units='mm') mp.postprocess(filename='test.png')
def main(): """Go Main Go.""" years = 12.0 # 2008 - 2019 pgconn = get_dbconn("idep") postgis = get_dbconn("postgis") # Get the initial geometries df = read_postgis( """ SELECT ugc, name, geom from ugcs WHERE end_ts is null and substr(ugc, 1, 3) = 'IAC' """, postgis, index_col="ugc", crs="EPSG:4326", ) scenario = 0 df2 = read_sql( """WITH data as ( SELECT r.huc_12, sum(avg_loss) * 4.163 / %s as detach, sum(avg_delivery) * 4.163 / %s as delivery, sum(avg_runoff) / 25.4 / %s as runoff from results_by_huc12 r , huc12 h WHERE r.huc_12 = h.huc_12 and h.states ~* 'IA' and r.scenario = %s and h.scenario = 0 and r.valid < '2020-01-01' and r.valid > '2008-01-01' GROUP by r.huc_12) SELECT ugc, avg(detach) as detach, avg(delivery) as delivery, avg(runoff) as runoff from data d JOIN huc12 h on (d.huc_12 = h.huc_12) WHERE h.scenario = 0 GROUP by ugc ORDER by delivery desc """, pgconn, params=(years, years, years, scenario), index_col="ugc", ) newcols = { "detach": "det%s" % (0, ), "delivery": "del%s" % (0, ), "runoff": "run%s" % (0, ), } for key, val in newcols.items(): df[val] = df2[key] df = df.sort_values("del0", ascending=False) print(df.head(10)) mp = MapPlot(title="2008-2019 DEP Top 10 Erosive Counties", logo="dep", caption="") df2 = df.head(10) mp.fill_ugcs(df2["del0"].to_dict()) mp.postprocess(filename="test.png")
def main(): """Go Main Go.""" pgconn = get_dbconn("idep") df = read_postgis( """ with centroids as ( select huc_12, st_centroid(geom) as center, simple_geom from huc12 where scenario = 0), agg as ( select c.huc_12, sum(case when st_y(center) < st_ymax(geom) then 1 else 0 end) as west, count(*) from flowpaths f JOIN centroids c on (f.huc_12 = c.huc_12) WHERE f.scenario = 0 GROUP by c.huc_12) select a.huc_12, st_transform(c.simple_geom, 4326) as geo, a.west, a.count from agg a JOIN centroids c ON (a.huc_12 = c.huc_12) """, pgconn, index_col=None, geom_col="geo", ) df["percent"] = df["west"] / df["count"] * 100.0 bins = np.arange(0, 101, 10) cmap = plt.get_cmap("RdBu") norm = mpcolors.BoundaryNorm(bins, cmap.N) mp = MapPlot( continentalcolor="thistle", nologo=True, sector="custom", south=36.8, north=48.0, west=-99.2, east=-88.9, subtitle="", title=("DEP Flowpaths North of HUC12 Centroid (%.0f/%.0f %.2f%%)" % ( df["west"].sum(), df["count"].sum(), df["west"].sum() / df["count"].sum() * 100.0, )), ) for _i, row in df.iterrows(): c = cmap(norm([row["percent"]]))[0] arr = np.asarray(row["geo"].exterior) points = mp.ax.projection.transform_points(ccrs.Geodetic(), arr[:, 0], arr[:, 1]) p = Polygon(points[:, :2], fc=c, ec="None", zorder=2, lw=0.1) mp.ax.add_patch(p) mp.drawcounties() mp.draw_colorbar(bins, cmap, norm, title="Percent", extend="neither") mp.postprocess(filename="/tmp/huc12_north.png")
def main(): """Go Main Go.""" pgconn = get_dbconn("idep") df = read_postgis( """ select f.huc_12, count(*) as fps, st_transform(h.simple_geom, 4326) as geo from flowpaths f JOIN huc12 h on (f.huc_12 = h.huc_12) WHERE f.scenario = 0 and h.scenario = 0 GROUP by f.huc_12, geo ORDER by fps ASC """, pgconn, index_col=None, geom_col="geo", ) bins = np.arange(1, 42, 2) cmap = plt.get_cmap("copper") cmap.set_over("white") cmap.set_under("thistle") norm = mpcolors.BoundaryNorm(bins, cmap.N) mp = MapPlot( continentalcolor="thistle", nologo=True, sector="custom", south=36.8, north=45.0, west=-99.2, east=-88.9, subtitle="", title=("DEP HUCs with <40 Flowpaths (%.0f/%.0f %.2f%%)" % ( len(df[df["fps"] < 40].index), len(df.index), len(df[df["fps"] < 40].index) / len(df.index) * 100.0, )), ) for _i, row in df.iterrows(): c = cmap(norm([row["fps"]]))[0] arr = np.asarray(row["geo"].exterior) points = mp.ax.projection.transform_points(ccrs.Geodetic(), arr[:, 0], arr[:, 1]) p = Polygon(points[:, :2], fc=c, ec="None", zorder=2, lw=0.1) mp.ax.add_patch(p) mp.drawcounties() mp.draw_colorbar(bins, cmap, norm, title="Count") mp.postprocess(filename="/tmp/huc12_cnts.png")
def plot_gdd(ts): """Generate our plot.""" nc = ncopen(ts.strftime("/mesonet/data/ndfd/%Y%m%d%H_ndfd.nc")) # compute our daily GDDs gddtot = np.zeros(np.shape(nc.variables["lon"][:])) for i in range(7): gddtot += gdd( temperature(nc.variables["high_tmpk"][i, :, :], "K"), temperature(nc.variables["low_tmpk"][i, :, :], "K"), ) cnc = ncopen("/mesonet/data/ndfd/ndfd_dailyc.nc") offset = daily_offset(ts) avggdd = np.sum(cnc.variables["gdd50"][offset:offset + 7], 0) data = gddtot - np.where(avggdd < 1, 1, avggdd) subtitle = ("Based on National Digital Forecast Database (NDFD) " "00 UTC Forecast made %s") % (ts.strftime("%-d %b %Y"), ) mp = MapPlot( title="NWS NDFD 7 Day (%s through %s) GDD50 Departure from Avg" % ( ts.strftime("%-d %b"), (ts + datetime.timedelta(days=6)).strftime("%-d %b"), ), subtitle=subtitle, sector="iailin", ) mp.pcolormesh( nc.variables["lon"][:], nc.variables["lat"][:], data, np.arange(-80, 81, 20), cmap=plt.get_cmap("RdBu_r"), units=r"$^\circ$F", spacing="proportional", ) mp.drawcounties() pqstr = ( "data c %s summary/cb_ndfd_7day_gdd.png summary/cb_ndfd_7day_gdd.png " "png") % (ts.strftime("%Y%m%d%H%M"), ) mp.postprocess(pqstr=pqstr) mp.close() nc.close()
def plotter(ctx): """ Go """ # Covert datetime to UTC do_polygon(ctx) m = MapPlot( title='2009-2018 Flash Flood Emergency Polygon Heatmap', sector='custom', axisbg='white', # west=-107, south=25.5, east=-88, north=41, # west=-82, south=36., east=-68, north=48, west=-85, south=31.8, north=45.2, east=-69, subtitle='based on unofficial IEM Archives', nocaption=True) cmap = plt.get_cmap('jet') cmap.set_under('white') cmap.set_over('black') res = m.pcolormesh(ctx['lons'], ctx['lats'], ctx['data'], ctx['bins'], cmap=cmap, units='count') # Cut down on SVG et al size res.set_rasterized(True) m.postprocess(filename='test.png')
def main(): """Go Main Go.""" dbconn = get_dbconn("postgis") df = read_sql(""" WITH data as ( SELECT wfo, eventid, extract(year from issue) as year, max(case when svs is not null then 1 else 0 end) as hit from warnings where product_issue > '2014-04-01' and product_issue < '2019-02-22' and phenomena = 'SV' and date(issue) not in ('2017-08-25', '2017-08-26', '2017-08-27', '2017-08-28', '2017-08-29', '2017-08-30') and significance = 'W' GROUP by wfo, eventid, year ) SELECT wfo, sum(hit) as got_update, count(*) as total_events from data GROUP by wfo ORDER by total_events DESC """, dbconn, index_col='wfo') if 'JSJ' in df.index and 'SJU' not in df.index: df.loc['SJU'] = df.loc['JSJ'] df['no_update_percent'] = ( 100. - df['got_update'] / df['total_events'] * 100. ) df.to_csv("140401_190221_svr_nofls.csv") # NOTE: FFW followup is FFS mp = MapPlot( sector='nws', title='Percentage of Severe TStorm Warnings without a SVS Update Issued', subtitle=('1 April 2014 - 21 February 2019 (exclude Harvey 26-30 Aug ' '2017), based on unofficial data') ) cmap = plt.get_cmap("copper_r") cmap.set_under('white') cmap.set_over('black') ramp = range(0, 101, 5) mp.fill_cwas( df['no_update_percent'], bins=ramp, cmap=cmap, units='%', ilabel=True, lblformat='%.1f' ) mp.postprocess(filename='140401_190221_svr_nosvs.png')
def main(): """Go Main Go""" data = get_data() mp = MapPlot(sector='midwest', title='8 July 2018 USDA NASS Corn Progress Percent Silking', subtitle=('Top value is 2018 percentage, bottom value is ' 'departure from 2008-2017 avg')) data2 = {} labels = {} for state in data: val = data[state]['d2017'] - data[state]['avg'] data2[state] = val labels[state.encode('utf-8')] = "%i%%\n%s%.1f%%" % (data[state]['d2017'], "+" if val > 0 else "", val) print(labels) levels = range(-40, 41, 10) mp.fill_states(data2, ilabel=True, labels=labels, bins=levels, cmap=plt.get_cmap('RdBu_r'), units='Absolute %', labelfontsize=16) mp.postprocess(filename='test.png') mp.close()
def main(): """Go Main Go.""" pgconn = get_dbconn('idep') df = read_postgis(""" select f.huc_12, count(*) as fps, st_transform(h.simple_geom, 4326) as geo from flowpaths f JOIN huc12 h on (f.huc_12 = h.huc_12) WHERE f.scenario = 0 and h.scenario = 0 GROUP by f.huc_12, geo ORDER by fps ASC """, pgconn, index_col=None, geom_col='geo') bins = np.arange(1, 42, 2) cmap = plt.get_cmap('copper') cmap.set_over('white') cmap.set_under('thistle') norm = mpcolors.BoundaryNorm(bins, cmap.N) mp = MapPlot( continentalcolor='thistle', nologo=True, sector='custom', south=36.8, north=45.0, west=-99.2, east=-88.9, subtitle='', title=('DEP HUCs with <40 Flowpaths (%.0f/%.0f %.2f%%)' % ( len(df[df['fps'] < 40].index), len(df.index), len(df[df['fps'] < 40].index) / len(df.index) * 100. ))) for _i, row in df.iterrows(): c = cmap(norm([row['fps'], ]))[0] arr = np.asarray(row['geo'].exterior) points = mp.ax.projection.transform_points( ccrs.Geodetic(), arr[:, 0], arr[:, 1]) p = Polygon(points[:, :2], fc=c, ec='None', zorder=2, lw=0.1) mp.ax.add_patch(p) mp.drawcounties() mp.draw_colorbar( bins, cmap, norm, title='Count') mp.postprocess(filename='/tmp/huc12_cnts.png')
def makeplot(ts, routes='ac'): """ Generate two plots for a given time GMT """ pgconn = get_dbconn('smos', user='******') df = read_sql(""" WITH obs as ( SELECT grid_idx, avg(soil_moisture) * 100. as sm, avg(optical_depth) as od from data where valid BETWEEN %s and %s GROUP by grid_idx) SELECT ST_x(geom) as lon, ST_y(geom) as lat, CASE WHEN sm is Null THEN -1 ELSE sm END as sm, CASE WHEN od is Null THEN -1 ELSE od END as od from obs o JOIN grid g ON (o.grid_idx = g.idx) """, pgconn, params=(ts - datetime.timedelta(hours=6), ts + datetime.timedelta(hours=6)), index_col=None) if df.empty: print(("Did not find SMOS data for: %s-%s" ) % (ts - datetime.timedelta(hours=6), ts + datetime.timedelta(hours=6))) return for sector in ['midwest', 'iowa']: clevs = np.arange(0, 71, 5) mp = MapPlot(sector=sector, axisbg='white', title='SMOS Satellite: Soil Moisture (0-5cm)', subtitle="Satelite passes around %s UTC" % ( ts.strftime("%d %B %Y %H"),)) if sector == 'iowa': mp.drawcounties() cmap = cm.get_cmap('jet_r') cmap.set_under('#EEEEEE') cmap.set_over("k") mp.hexbin(df['lon'].values, df['lat'].values, df['sm'], clevs, units='%', cmap=cmap) pqstr = "plot %s %s00 smos_%s_sm%s.png smos_%s_sm%s.png png" % ( routes, ts.strftime("%Y%m%d%H"), sector, ts.strftime("%H"), sector, ts.strftime("%H")) mp.postprocess(pqstr=pqstr) mp.close() for sector in ['midwest', 'iowa']: clevs = np.arange(0, 1.001, 0.05) mp = MapPlot(sector=sector, axisbg='white', title=('SMOS Satellite: Land Cover Optical Depth ' '(microwave L-band)'), subtitle="Satelite passes around %s UTC" % ( ts.strftime("%d %B %Y %H"),)) if sector == 'iowa': mp.drawcounties() cmap = cm.get_cmap('jet') cmap.set_under('#EEEEEE') cmap.set_over("k") mp.hexbin(df['lon'].values, df['lat'].values, df['od'], clevs, cmap=cmap) pqstr = "plot %s %s00 smos_%s_od%s.png smos_%s_od%s.png png" % ( routes, ts.strftime("%Y%m%d%H"), sector, ts.strftime("%H"), sector, ts.strftime("%H")) mp.postprocess(pqstr=pqstr) mp.close()
def makeplot(ts, routes="ac"): """ Generate two plots for a given time GMT """ pgconn = get_dbconn("smos", user="******") df = read_sql( """ WITH obs as ( SELECT grid_idx, avg(soil_moisture) * 100. as sm, avg(optical_depth) as od from data where valid BETWEEN %s and %s GROUP by grid_idx) SELECT ST_x(geom) as lon, ST_y(geom) as lat, CASE WHEN sm is Null THEN -1 ELSE sm END as sm, CASE WHEN od is Null THEN -1 ELSE od END as od from obs o JOIN grid g ON (o.grid_idx = g.idx) """, pgconn, params=( ts - datetime.timedelta(hours=6), ts + datetime.timedelta(hours=6), ), index_col=None, ) if df.empty: LOG.info( "Did not find SMOS data for: %s-%s", ts - datetime.timedelta(hours=6), ts + datetime.timedelta(hours=6), ) return for sector in ["midwest", "iowa"]: clevs = np.arange(0, 71, 5) mp = MapPlot( sector=sector, axisbg="white", title="SMOS Satellite: Soil Moisture (0-5cm)", subtitle="Satelite passes around %s UTC" % (ts.strftime("%d %B %Y %H"), ), ) if sector == "iowa": mp.drawcounties() cmap = get_cmap("jet_r") cmap.set_under("#EEEEEE") cmap.set_over("k") mp.hexbin( df["lon"].values, df["lat"].values, df["sm"], clevs, units="%", cmap=cmap, ) pqstr = "plot %s %s00 smos_%s_sm%s.png smos_%s_sm%s.png png" % ( routes, ts.strftime("%Y%m%d%H"), sector, ts.strftime("%H"), sector, ts.strftime("%H"), ) mp.postprocess(pqstr=pqstr) mp.close() for sector in ["midwest", "iowa"]: clevs = np.arange(0, 1.001, 0.05) mp = MapPlot( sector=sector, axisbg="white", title=("SMOS Satellite: Land Cover Optical Depth " "(microwave L-band)"), subtitle="Satelite passes around %s UTC" % (ts.strftime("%d %B %Y %H"), ), ) if sector == "iowa": mp.drawcounties() cmap = get_cmap("jet") cmap.set_under("#EEEEEE") cmap.set_over("k") mp.hexbin(df["lon"].values, df["lat"].values, df["od"], clevs, cmap=cmap) pqstr = "plot %s %s00 smos_%s_od%s.png smos_%s_od%s.png png" % ( routes, ts.strftime("%Y%m%d%H"), sector, ts.strftime("%H"), sector, ts.strftime("%H"), ) mp.postprocess(pqstr=pqstr) mp.close()