matplotlib Basemaps with the ``MapDesign`` class and how to plot ``Trajectory`` paths and color according to a single value. Here, we'll get into more advanced usage of ``MapDesign`` and learn a couple of ways to scatter plot our along-trajectory data. For this example we'll initialize only the February trajectories created in ``reversetraj_clippedtraj_gen.py``. """ import matplotlib.pyplot as plt import numpy as np import pysplit trajgroup = pysplit.make_trajectorygroup(r'C:/trajectories/umn_example/*feb*') """ Basemaps and Advanced MapDesign ------------------------------- We reiterate here that PySPLIT can plot on any matplotlib Basemap, but the ``MapDesign`` class of PySPLIT can expedite setting up Basemaps. A basic cylindrical map using ``MapDesign`` only requires ``mapcorners`` and ``standard_pm=None``. However, we can adjust the map projection, label spacing, arrangement, and sizes; how and which boundaries are drawn, even switch up the background colors between three different gray level sets, among other actions. Basemaps can be drawn on new or existing axes. Let's make a figure with two Lambert Conformal Conic maps with states and medium gray backgrounds. We'll initialize two ``MapDesign``s, since we want
""" ============================================= Basic Trajectory Plotting and Using MapDesign ============================================= How to quickly initialize a matplotlib Basemap with the ``MapDesign`` class and plot ``Trajectory`` paths. For this example we'll initialize only the January trajectories created in ``bulk_trajgen_example.py``. """ import pysplit trajgroup = pysplit.make_trajectorygroup(r'C:/trajectories/colgate/*jan*') """ Basemaps and MapDesign ---------------------- PySPLIT's ``MapDesign`` class uses the matplotlib Basemap toolkit to quickly set up attractive maps. The user is not restricted to using maps produced from ``MapDesign``, however- any Basemap will serve in the section below entitled 'Plotting ``Trajectory`` Paths. Creating a basic cylindrical map using ``MapDesign`` only requires ``mapcorners``, a list of the lower-left longitude, lower-left latitude, upper-right longitude, and upper-right latitude values. The ``standard_pm``, a list of standard parallels and meridians, may be passed as ``None``. """
Below, we'll generate back trajectories from the University of Minnesota,
forgoing the "reverse" and "clipped" trajectories (``get_reverse=False``,
``get_clipped=False``). For a detailed explanation of this call, see
``bulk_trajgen_example.py``. Then, initialize a TrajectoryGroup.
"""
from __future__ import print_function
import pysplit
pysplit.generate_bulktraj('umn', r'C:/hysplit4/working',
r'C:/trajectories/umn_example', r'E:/gdas', [2013],
range(1,4), [6, 15, 18, 21], [1000], (44.97, -93.23),
-96, get_clipped=False)
trajgroup = pysplit.make_trajectorygroup(r'C:/trajectories/umn_example/*')
"""
In the course of our analysis, we might decide that a subset of trajectories
is more appropriate for our purpose. Let's create a new ``TrajectoryGroup``
with all the trajectories with a temperature at t=0 of 0 degrees C and greater.
"""
warm_trajlist = [traj for traj in trajgroup if traj.data.Temperature_C[0] > 0.0]
warm_trajgroup = pysplit.TrajectoryGroup(warm_trajlist)
"""
The new ``TrajectoryGroup`` is much smaller:
"""
def gen_trajs(iso_18,
xy_df_for_hysplit,
month_real,
altitude,
points_all_in_water_report,
points_origin_not_detected_report,
error_in_meteo_file,
traj_shorter_than_runtime_report,
input_pass_to_bulkrajfun_report,
Sampling_date_db=False):
working_dir = r'C:/hysplit4/working'
meteo_dir = r'C:/Users/Ash kan/Documents/meteo_iso_model/meteo_iso_model_input_code_and_results/inputs_sonia_paper/meteo_for_trajs_pysplit/meteo_for_traj_new_name' #.gbl files : ex: RPapr1973.gbl format basename+3 letter month + year
basename = 'RP'
hours = [12]
altitudes = [altitude]
runtime = -350
all_hysplit = list()
nw_ex_all = list()
cols = [
"ID_MeteoPoint", "newLat", "newLong", "real_distt_alt_n" + "_" +
str(altitude), "real_distt_alt_s" + "_" + str(altitude),
"real_distt_pac_s" + "_" + str(altitude), "real_distt_pac_n" + "_" +
str(altitude), "straight_distt_alt_n" + "_" + str(altitude),
"straight_distt_alt_s" + "_" + str(altitude), "straight_distt_pac_s" +
"_" + str(altitude), "straight_distt_pac_n" + "_" + str(altitude),
"percentage_alt_n" + "_" + str(altitude), "percentage_alt_s" + "_" +
str(altitude), "percentage_pac_s" + "_" + str(altitude),
"percentage_pac_n" + "_" + str(altitude), "num_of_all_trajs" + "_" +
str(altitude), "num_of_unknown_origins" + "_" + str(altitude)
]
#making alt folders
storage_dir1 = os.path.join(r'C:\trajectories',
basename + "\\" + str(altitude))
if not os.path.isdir(storage_dir1):
os.mkdir(storage_dir1)
######################
###########################
for index, row in xy_df_for_hysplit.iterrows():
storage_dir = os.path.join(storage_dir1, str(row["ID_MeteoPoint"]))
if not os.path.isdir(storage_dir):
os.mkdir(storage_dir)
storage_dir = os.path.join(storage_dir, str(month_real))
################
location = (row["newLat"], row["newLong"])
tem = iso_18.loc[iso_18["CooX"] == row["CooX"]]
temp = tem.loc[tem["CooY"] == row["CooY"]]
#new code to consider database for real dates! 30 april 2020
if Sampling_date_db == False:
dates_general = pd.DatetimeIndex(temp["Date"])
dates_general = dates_general[dates_general.month.isin(
[month_real])]
years = dates_general.year.to_list()
months = dates_general.month.to_list()
years = list(dict.fromkeys(years))
months = list(dict.fromkeys(months))
days = None
day_type = "all_month"
pysplit.generate_bulktraj(
basename,
working_dir,
storage_dir,
meteo_dir,
years,
months,
hours,
altitudes,
location,
runtime,
error_in_meteo_file=error_in_meteo_file,
days=days, #list or none
day_type=day_type, #"all_month" or "daily"
meteo_bookends=([1, 2, 3, 4, 5], [1]),
monthslice=slice(0, 32, 1),
meteoyr_2digits=False,
get_reverse=False,
get_clipped=False)
elif Sampling_date_db == True:
dates_for_daily_trajs = pd.concat(
[temp["Date"], temp["Date_Meas_real"], temp["day_type"]],
names=["Date", "Date_Meas_real", "day_type"],
axis=1)
dates_for_daily_trajs = dates_for_daily_trajs[pd.DatetimeIndex(
dates_for_daily_trajs["Date"]).month.isin([month_real])]
dates_general = pd.DatetimeIndex(dates_for_daily_trajs["Date"])
dates_measured = pd.DatetimeIndex(
dates_for_daily_trajs["Date_Meas_real"])
if "Monthly" in dates_for_daily_trajs["day_type"]:
years = dates_general.year.to_list()
months = dates_general.month.to_list()
years = list(dict.fromkeys(years))
months = list(dict.fromkeys(months))
days = None
day_type = "all_month"
pysplit.generate_bulktraj(
basename,
working_dir,
storage_dir,
meteo_dir,
years,
months,
hours,
altitudes,
location,
runtime,
error_in_meteo_file=error_in_meteo_file,
days=days, #list or none
day_type=day_type, #"all_month" or "daily"
meteo_bookends=([1, 2, 3, 4, 5], [1]),
monthslice=slice(0, 32, 1),
meteoyr_2digits=False,
get_reverse=False,
get_clipped=False)
else: #day_type="daily"
date_measured_done_list = list()
for date in dates_measured:
if date not in date_measured_done_list:
date_measured_done_list.append(date)
years = [date.year]
months = [date.month]
days = [date.day]
day_type = "daily"
days = list(dict.fromkeys(days))
years = list(dict.fromkeys(years))
months = list(dict.fromkeys(months))
###############################
#print_for_input_pass_to_bulkrajfun_report="\n\n##################\n"+str(altitudes)+"\n"+str(row["ID_MeteoPoint"])+"\n"+str(months)+"\n"+str(years)+"\n"+str(days)
#print (print_for_input_pass_to_bulkrajfun_report)
###############################
pysplit.generate_bulktraj(
basename,
working_dir,
storage_dir,
meteo_dir,
years,
months,
hours,
altitudes,
location,
runtime,
error_in_meteo_file=error_in_meteo_file,
days=days, #list or none
day_type=day_type, #"all_month" or "daily"
meteo_bookends=([1, 2, 3, 4, 5], [1]),
monthslice=slice(0, 32, 1),
meteoyr_2digits=False,
get_reverse=False,
get_clipped=False)
if len(os.listdir(storage_dir)) == 0:
print("remove empty folder!")
shutil.rmtree(storage_dir)
continue
storage_dir_for_plot = os.path.join(storage_dir, 'plots')
if not os.path.isdir(storage_dir_for_plot):
os.mkdir(storage_dir_for_plot)
storage_dir = os.path.join(storage_dir, '*')
print("storage_dir", storage_dir)
umn_tg = pysplit.make_trajectorygroup(storage_dir)
bm = Basemap() # default: projection='cyl'
real_traj_dist_list_all_years_all_hours = list()
straight_distt_alt_n = 0
straight_distt_alt_s = 0
straight_distt_pac_s = 0
straight_distt_pac_n = 0
cnt_alt_n = 0
cnt_alt_s = 0
cnt_pac_s = 0
cnt_pac_n = 0
real_distt_alt_n = 0
real_distt_alt_s = 0
real_distt_pac_s = 0
real_distt_pac_n = 0
cnt_unk = 0
for traj in umn_tg:
timestep_is_border = -1 * len(traj.data.geometry)
#sometimes hysplit trajectories is not complete because z in coordination reaches the ground!
'''if len(traj.data.geometry)<101:
#write it to a report file
traj_shorter_than_runtime_report.write("\n#####Trajectory data#####\n")
traj_shorter_than_runtime_report.write(str(traj.data.loc[0]))
traj_shorter_than_runtime_report.write("\n")
print("traj_shorter_than_runtime")
continue'''
for i in range(len(traj.data.geometry) - 1, -1, -1):
timestep_is_border = timestep_is_border + 1
lng = traj.data.geometry[-i].x
lat = traj.data.geometry[-i].y
if (lng > -100 and
lng < -32) and (lat > -60 and lat < 14) and bm.is_land(
lng, lat) == True: #True or False
#print ("timestep_is_border:",timestep_is_border)
#to consider the point in water! (Since there are points that the first reverse step goes into water!)
if timestep_is_border == -1 * len(traj.data.geometry) + 1:
pass
else:
timestep_is_border = timestep_is_border - 1
break
if timestep_is_border == 0:
print("points_all_in_water")
points_all_in_water_report.write(
"\n\n##########\n##########\n\n" +
str(traj.data.loc[timestep_is_border]) + "\n\n")
continue
#print ("ATTENTION, Still in land, more timesteps!!!!")
#else:
# timestep_is_border=timestep_is_border+1
#print ("timestep_is_water",timestep_is_border)
'''traj.generate_reversetraj(working_dir, meteo_dir,run=timestep_is_border,
reverse_dir='default',
meteo_interval='monthly',
hysplit="C:\\hysplit4\\exec\\hyts_std")'''
#traj.load_reversetraj()
#traj.calculate_distance(reverse=True)
traj.calculate_distance()
#print("len",len(traj.data.geometry))
long_coast = traj.data.loc[timestep_is_border].geometry.x
lat_coast = traj.data.loc[timestep_is_border].geometry.y
if long_coast <= -32 and long_coast >= -69 and lat_coast >= -15 and lat_coast <= 14: #ok
continentality = "atl_n"
cnt_alt_n = cnt_alt_n + 1
straight_distt_alt_n = straight_distt_alt_n + traj.data.loc[
timestep_is_border]["Dist_from_origin"]
real_distt_alt_n = real_distt_alt_n + traj.data.loc[
timestep_is_border]["Cumulative_Dist"]
elif long_coast <= -32 and long_coast >= -69 and lat_coast >= -60 and lat_coast <= -15: #ok
continentality = "atl_s"
cnt_alt_s = cnt_alt_s + 1
straight_distt_alt_s = straight_distt_alt_s + traj.data.loc[
timestep_is_border]["Dist_from_origin"]
real_distt_alt_s = real_distt_alt_s + traj.data.loc[
timestep_is_border]["Cumulative_Dist"]
elif long_coast <= -69 and long_coast >= -100 and lat_coast >= -60 and lat_coast <= -15:
continentality = "pac_s"
cnt_pac_s = cnt_pac_s + 1
straight_distt_pac_s = straight_distt_pac_s + traj.data.loc[
timestep_is_border]["Dist_from_origin"]
real_distt_pac_s = real_distt_pac_s + traj.data.loc[
timestep_is_border]["Cumulative_Dist"]
elif long_coast <= -69 and long_coast >= -100 and lat_coast >= -15 and lat_coast <= 14:
continentality = "pac_n"
cnt_pac_n = cnt_pac_n + 1
straight_distt_pac_n = straight_distt_pac_n + traj.data.loc[
timestep_is_border]["Dist_from_origin"]
real_distt_pac_n = real_distt_pac_n + traj.data.loc[
timestep_is_border]["Cumulative_Dist"]
else:
continentality = "UNKNOWN"
cnt_unk = cnt_unk + 1
points_origin_not_detected_report.write(
"\n\n##### First point #####\n\n" +
str(traj.data.loc[0].geometry) + "\n\n" +
str(traj.data.loc[0].DateTime) +
"\n\n#####last point #####\n\n" + str(timestep_is_border) +
"\n\n\n" + str(traj.data.loc[timestep_is_border]))
print("Attention: couldn't find the origin, skipping!",
timestep_is_border)
#18 june 2020 Ashkan: Sonia needs excel with trajctory data before averaging!!
if day_type == "daily":
nw_ex = [
row["ID_MeteoPoint"], row["newLat"], row["newLong"]
] + [traj.data.loc[timestep_is_border]["Cumulative_Dist"]] + [
traj.data.loc[timestep_is_border]["Dist_from_origin"]
] + [continentality] + altitudes + [
int(traj.trajid[-13:-9])
] + [int(traj.trajid[-9:-7])] + [int(traj.trajid[-7:-5])]
nw_ex_all.append(nw_ex)
#23 june 2020 Ashkan: Drawing traj maps
m = Basemap(projection='cyl',
llcrnrlat=-60,
urcrnrlat=30,
llcrnrlon=-90,
urcrnrlon=-20,
resolution='c')
m.drawcoastlines()
m.drawparallels(np.arange(-60., 31., 20.),
labels=[False, True, True, False])
m.drawmeridians(np.arange(-90., -21., 20.),
labels=[True, False, False, True])
plt.plot(*traj.path.xy, color="c")
pltname = os.path.join(
storage_dir_for_plot,
str(row["ID_MeteoPoint"]) + "_" + traj.trajid[-13:-3])
plt.savefig(pltname, dpi=300)
plt.close()
#######################################
average_real_dist = list()
average_straight_distt = list()
porc_list = list()
cnt_sum = (cnt_alt_n + cnt_alt_s + cnt_pac_n + cnt_pac_s)
real_dist_list = [
real_distt_alt_n, real_distt_alt_s, real_distt_pac_s,
real_distt_pac_n
]
str_dist_list = [
straight_distt_alt_n, straight_distt_alt_s, straight_distt_pac_s,
straight_distt_pac_n
]
cnt_list = [cnt_alt_n, cnt_alt_s, cnt_pac_s, cnt_pac_n]
for real_dist, str_dist, cnt in zip(real_dist_list, str_dist_list,
cnt_list):
porc_list.append(cnt / cnt_sum)
try:
average_real_dist.append(real_dist / cnt)
average_straight_distt.append(str_dist / cnt)
except:
average_real_dist.append(0)
average_straight_distt.append(0)
te = [row["ID_MeteoPoint"], row["newLat"], row["newLong"]
] + average_real_dist + average_straight_distt + porc_list + [
cnt_sum
] + [cnt_unk]
all_hysplit.append(te)
all_hysplit_df = pd.DataFrame(all_hysplit, columns=cols)
#nw_ex_all_df=pd.DataFrame(nw_ex_all,columns=["ID_MeteoPoint","newLat", "newLong","Cumulative_Dist","Dist_from_origin","continentality","altitude","year","month","day"])
nw_ex_all_df = nw_ex_all
#all_hysplit_df.to_excel(r"C:\Users\Ash kan\Documents\meteo_iso_model\meteo_iso_model_input_code_and_results\output\all_hysplit.xls")
return all_hysplit_df, nw_ex_all_df
Basic Trajectory Plotting and Using MapDesign
=============================================
How to quickly initialize a matplotlib Basemap with the ``MapDesign``
class and plot ``Trajectory`` paths.
For this example we'll initialize only the January trajectories created in
``bulk_trajgen_example.py``.
"""
# %%
import pysplit
# %%
trajgroup = pysplit.make_trajectorygroup(
'/Users/diego/pysplit/example_data/Archive/*')
# %%
"""
Basemaps and MapDesign
----------------------
PySPLIT's ``MapDesign`` class uses the matplotlib Basemap toolkit to quickly
set up attractive maps. The user is not restricted to using maps
produced from ``MapDesign``, however- any Basemap will serve in the section
below entitled 'Plotting ``Trajectory`` Paths.
Creating a basic cylindrical map using ``MapDesign`` only requires
``mapcorners``, a list of the lower-left longitude, lower-left latitude,
upper-right longitude, and upper-right latitude values.
The ``standard_pm``, a list of standard parallels and meridians,
may be passed as ``None``.
"""
============================
Estimating Integration Error
============================
"""
import numpy as np
import pysplit
import os
trajgroup = pysplit.make_trajectorygroup(r'C:/Users/User/Desktop/londonclim/*')
##creating a trajectorygroup containing BTs for calculation
for traj in trajgroup:
traj.load_reversetraj()
##loading the corresponding reverse trajectories in "traj"
for traj in trajgroup:
traj.calculate_integrationerr()
##calulating integration error
relative_errors = [traj.integration_error for traj in trajgroup]
cutoff = np.mean(relative_errors) + (np.std(relative_errors) * 2)
bad = []
##creating cutoff to separate"bad" trajectories (with int errors larger than two stdev from the mean)
for traj in trajgroup:
if traj.integration_error > cutoff:
bad.append(traj.trajid)
##selecting "bad" trajectories and creating a new trajgroup without bad trajectories
print('Expectation: ', trajgroup.trajcount, 'trajectories -', len(bad),
'bad trajectories =', trajgroup.trajcount - len(bad), 'trajectories')
trajgroup.pop(trajid=bad)
print('Result: ', trajgroup.trajcount, 'trajectories')
##visualising the number of initial trajectories, minus the bad trajectories and the result (trajectories to use)
print(bad)
------------------------
We begin by creating a ``TrajectoryGroup`` with the desired trajectories.
In this example, we will use the back trajectories generated in
``bulk_trajgen_example.py``. For this experiment, we decide in advance to only
use the August trajectories initialized at 1000 m above ground at noon.
"""
from __future__ import division
import numpy as np
import matplotlib.pyplot as plt
import pysplit
# We ignored Daylight Saving Time during generation. EST is UTC-5
trajgroup = pysplit.make_trajectorygroup(
r'C:/trajectories/colgate/*aug1000*17')
"""
Next, we create the list of trajectory files for HYSPLIT to cluster. The
required argument, ``infile_dir``, is the directory in which this 'INFILE'
will be saved and in which HYSPLIT will place all new files associated with
this run. By default, ``use_clippedpath=True``. If clipped trajectories are
not found then PySPLIT will output the paths of the original trajectory files.
"""
trajgroup.make_infile(r'C:/hysplit4/cluster/working')
"""
Clustering
----------
In the HYSPLIT GUI, navigate to the standard clustering run.
Under 'Step 1: Inputs', the user may adjust the clustering parameters (hours,
First, reverse trajectories must be available. For information on how to
generate reverse trajectories with ``PySPLIT``, see ``bulk_trajgen_example.py``
and ``reversetraj_clippedtraj_gen.py``.
Setup
-----
Load the original and reverse trajectories. This example uses the
trajectories generated in ``bulk_trajgen_example.py``.
"""
from __future__ import print_function
import numpy as np
import pysplit
trajgroup = pysplit.make_trajectorygroup(r'C:/trajectories/colgate/*')
for traj in trajgroup:
traj.load_reversetraj()
"""
Calculating integration error
-----------------------------
Values computed when calling ``Trajectory.calculate_integrationerr()``:
``Trajectory.integration_error``, the relative error (%)
``Trajectory.integration_error_abs``, the absolute error (meters)
"""
for traj in trajgroup:
traj.calculate_integrationerr()
"""
------------------------ We begin by creating a ``TrajectoryGroup`` with the desired trajectories. In this example, we will use the back trajectories generated in ``bulk_trajgen_example.py``. For this experiment, we decide in advance to only use the August trajectories initialized at 1000 m above ground at noon. """ from __future__ import division import numpy as np import matplotlib.pyplot as plt import pysplit # We ignored Daylight Saving Time during generation. EST is UTC-5 trajgroup = pysplit.make_trajectorygroup(r'C:/trajectories/colgate/*aug1000*17') """ Next, we create the list of trajectory files for HYSPLIT to cluster. The required argument, ``infile_dir``, is the directory in which this 'INFILE' will be saved and in which HYSPLIT will place all new files associated with this run. By default, ``use_clippedpath=True``. If clipped trajectories are not found then PySPLIT will output the paths of the original trajectory files. """ trajgroup.make_infile(r'C:/hysplit4/cluster/working') """ Clustering ---------- In the HYSPLIT GUI, navigate to the standard clustering run.
years = [2016,2017]
months = [4,4]
hours = [11,17,23]
altitudes = [1000, 3000, 5000]
location = (-54.62,158.85)
runtime = -72
pysplit.generate_bulktraj(basename, working_dir, storage_dir, meteo_dir,
years, months, hours, altitudes, location, runtime,
monthslice=slice(0, 32, 2), get_reverse=True,
get_clipped=True)
# %%
'''
basic plotting example
'''
trajgroup = pysplit.make_trajectorygroup('/Users/qingn/Desktop/NQ/radiosondes_micre/trajectories/trajector*')
"""
Basemaps and MapDesign
----------------------
PySPLIT's ``MapDesign`` class uses the matplotlib Basemap toolkit to quickly
set up attractive maps. The user is not restricted to using maps
produced from ``MapDesign``, however- any Basemap will serve in the section
below entitled 'Plotting ``Trajectory`` Paths.
Creating a basic cylindrical map using ``MapDesign`` only requires
``mapcorners``, a list of the lower-left longitude, lower-left latitude,
upper-right longitude, and upper-right latitude values.
The ``standard_pm``, a list of standard parallels and meridians,
may be passed as ``None``.
"""
mapcorners = [30, -80, 180, -30]
