Running with helper class¶
[This note-book is in oceantracker/tutorials_how_to/]
Oceantracker is designed so that both coders and non-coders can get almost the same level of adaptability to their specific needs. Also to streamline running many cases across distributed computers, along with enabling online particle tracking as a service.
To achieve these capabilities, it uses parameters to both change the settings to meet the users needs, but also to build the computational pipeline for the users specific needs. This flexibility in the computational pipeline is achieved using “class” parameters.
Classes perform specific roles in the pipeline. Parameters tell Oceantracker which version of core classes the user wants to use for each role, or add optional classes. Eg. add multiple “release_groups” classes, which may release particles at points, or within a polygon, at different times, rates and locations, all within the same computational run.
Simply by changing the parameters, users can load their own version of a class, including the “solver” class which orchestrates the time stepping and operations by other classes. There are base classes for all the major roles within the computational pipeline. Classes can be adapted by inheriting the bases class, or one of its children, and overwriting some of the methods to alter how the class performs its role.
To make the ideas of settings and classes easier to adopt, the below uses methods of a helper class to add settings and classes to the pipeline. The notebook E_run_using_parameter_dictionaries.ipynb, shows how run directly from parameter dictionaries, which are built in code or read from a json or yaml file.
Parameters using helper¶
There are two types of parameters:
Settings parameters
These are set with one or more calls to helper method, ot.setting(…), where … are keyword arguments. eg “time_step”, the model time step in seconds is set using, time_step= 1800.
There are default values for most settings. A full set of settings and their defaults is at ….
Class parameters
Classes add specific tasks to the computational pipeline, eg. how to release particles, write output, particle suspension etc. These are added using helper method ot.add_class(class_type, ….). The first argument is a “class_type”. Each class has its own specific settings, which are set using keyword arguments of helper method. Eg. ot.add_class(‘reader’, input_dir= ‘my_hindcast_dir’, ….) adds a reader class and tells is it which folder contains the hindcast files.
A full set of classes and their default settings is at ….
Normally there must be a keyword argument “class_name” setting. This is used to import the required class into the computational pipeline, with its specific settings. This “class_name” is a string, used to import the class at setup (so is the same as that used to import any python class within a module).
The “class_name” setting is normally required. A few class_types have a default class_name, eg. in minimal_example “release_groups” assumes a point_release, and the reader’s “class_name” is chosen by looking at variable names within the hindcast’s netcdf file.
There are two types of “class_type”s
When
Only a single class is required, these have singular “class_type”, eg.”reader” and “solver”.
One or more classes can be added. These have plural “class_type” (eg. can add multiple “release_groups”).
Documentation for all parameters/settings and their default values.
Add link here
Extend the minimal example¶
The below extends the minimal_example, it adds:
release at random locations within a polygon, at random water depths (the default)
a particle fall velocity
particles on the bottom are re-suspended if friction velocity exceeds a critical value.
# build parameters using helper class
from oceantracker.main import OceanTracker
ot = OceanTracker() # make an instance of the helper class
# one or more settings can be set by calls to os.settings
ot.settings(output_file_base='param_test1',# name used as base for output files
root_output_dir='output' # output is put in dir 'root_output_dir'\\'output_file_base'
)
# ot.settings can be used more than once to add more settings
ot.settings(time_step =120) # 2 min model time step as seconds
# add a compulsory reader class
# no class_name setting is required
# as will detect that it needs a a schism reader class
ot.add_class('reader',
input_dir= '..\\demos\\demo_hindcast', # folder to search for hindcast files, sub-dirs will, by default, also be searched
file_mask = 'demoHindcastSchism*.nc', # the file mask of the hindcast files
)
# add some release groups
# release_groups are one or more release groups
# (ie locations where particles are released at the same times and locations)
# there must be at least one release group
# add release locations from two points,
ot.add_class('release_groups',
name ='my_release_points1',
# the required name setting, is used to refer to this release group internally anf in postprocsing
points= [[1595000, 5482600],
[1599000, 5486200]], # must be an N by 2 or 3 or list, convertible to a numpy array
release_interval= 3600, # seconds between releasing particles
pulse_size = 10, # number of particles released each release_interval
# no class_name setting, so assumes a point release
)
# add a second release group, from random locations within a polygon
ot.add_class('release_groups',
name ='my_polygon_release1',
class_name= 'oceantracker.release_groups.polygon_release.PolygonRelease', # use a polygon release
# this time is a polygon , so below points are polygon cords
points = [ [1597682.1237, 5489972.7479],
[1598604.1667, 5490275.5488],
[1598886.4247, 5489464.0424],
[1597917.3387, 5489000],
[1597300, 5489000],
[1597682.1237, 5489972.7479]
],
release_interval= 7200, # seconds between releasing particles
pulse_size = 20, # number of particles released each release_interval
)
# alter default re-suspension class's default settings
ot.add_class('resuspension', critical_friction_velocity = .005) # only re-suspend particles if friction vel. exceeds this value
# add a class to modify the particle velocity
# velocity_modifiers are a set of velocities added to water velocity give in hydrodynamic model's
# here a fall velocity with given value is added to the computation
ot.add_class('velocity_modifiers',
name ='my_fall_velocity',
class_name = 'oceantracker.velocity_modifiers.terminal_velocity.TerminalVelocity',
mean= -0.001, # mean terminal vel < 0 for falling
# optionally variance can also be use to give each particles its own fall velocity
)
Running OceanTracker¶
There are several ways to run OceanTracker
By coding
user “class helper” method to build parameters (as above) then run
build parameters dictionary in code then run
read parameter file and then run
Without coding
run from command line with parameter file which is built by editing a json/yaml text file
Here we use the “class helper” method approach, the other ways to run directly using parameter dictionaries outlined in E_run_using_parameter_dictionaries.ipynb, add link…
Note:
There are many ways to run the code, eg. with IDE like Pycharm, Visual Studio Code. It can also, as here, be run in iPython notebooks. However the way notebooks are implemented can sometimes result in issues.
See … for more details
Below runs oceantracker using the helper class.
# first see the parameters build using the helper class instance
# this is a dictionary ot.params
print(ot.params) # ugly!
# use json.dumps() to make it pretty
import json
print(json.dumps(ot.params, indent=4))
# lots of params are shown
# as filling a template given by from user callable main.param_template()
# null or None are optional ones which will use defaults
{'add_date_to_run_output_dir': None, 'advanced_settings': {}, 'backtracking': None, 'block_dry_cells': None, 'case_output_file_tag': None, 'compact_mode': None, 'debug': None, 'duration': None, 'max_run_duration': None, 'minimum_total_water_depth': None, 'open_boundary_type': None, 'output_file_base': 'param_test1', 'particle_buffer_size': None, 'processors': None, 'retain_culled_part_locations': None, 'root_output_dir': 'output', 'run_as_depth_averaged': None, 'screen_output_time_interval': None, 'time_step': 120, 'user_note': None, 'write_grid': None, 'write_output_files': None, 'write_tracks': None, 'z0': None, 'dispersion': {}, 'field_group_manager': {}, 'interpolator': {}, 'particle_group_manager': {}, 'reader': {'input_dir': '..\demos\demo_hindcast', 'file_mask': 'demoHindcastSchism*.nc'}, 'resuspension': {'critical_friction_velocity': 0.005}, 'solver': {}, 'tracks_writer': {}, 'event_loggers': {}, 'fields': {}, 'particle_concentrations': {}, 'particle_properties': {}, 'particle_statistics': {}, 'release_groups': {'my_release_points1': {'points': [[1595000, 5482600], [1599000, 5486200]], 'release_interval': 3600, 'pulse_size': 10}, 'my_polygon_release1': {'class_name': 'oceantracker.particle_release_groups.polygon_release.PolygonRelease', 'points': [[1597682.1237, 5489972.7479], [1598604.1667, 5490275.5488], [1598886.4247, 5489464.0424], [1597917.3387, 5489000], [1597300, 5489000], [1597682.1237, 5489972.7479]], 'release_interval': 7200, 'pulse_size': 20}}, 'status_modifiers': {}, 'time_varying_info': {}, 'trajectory_modifiers': {}, 'velocity_modifiers': {'my_fall_velocity': {'class_name': 'oceantracker.velocity_modifiers.terminal_velocity.TerminalVelocity', 'mean': -0.001}}}
{
"add_date_to_run_output_dir": null,
"advanced_settings": {},
"backtracking": null,
"block_dry_cells": null,
"case_output_file_tag": null,
"compact_mode": null,
"debug": null,
"duration": null,
"max_run_duration": null,
"minimum_total_water_depth": null,
"open_boundary_type": null,
"output_file_base": "param_test1",
"particle_buffer_size": null,
"processors": null,
"retain_culled_part_locations": null,
"root_output_dir": "output",
"run_as_depth_averaged": null,
"screen_output_time_interval": null,
"time_step": 120,
"user_note": null,
"write_grid": null,
"write_output_files": null,
"write_tracks": null,
"z0": null,
"dispersion": {},
"field_group_manager": {},
"interpolator": {},
"particle_group_manager": {},
"reader": {
"input_dir": "..\demos\demo_hindcast",
"file_mask": "demoHindcastSchism*.nc"
},
"resuspension": {
"critical_friction_velocity": 0.005
},
"solver": {},
"tracks_writer": {},
"event_loggers": {},
"fields": {},
"particle_concentrations": {},
"particle_properties": {},
"particle_statistics": {},
"release_groups": {
"my_release_points1": {
"points": [
[
1595000,
5482600
],
[
1599000,
5486200
]
],
"release_interval": 3600,
"pulse_size": 10
},
"my_polygon_release1": {
"class_name": "oceantracker.particle_release_groups.polygon_release.PolygonRelease",
"points": [
[
1597682.1237,
5489972.7479
],
[
1598604.1667,
5490275.5488
],
[
1598886.4247,
5489464.0424
],
[
1597917.3387,
5489000
],
[
1597300,
5489000
],
[
1597682.1237,
5489972.7479
]
],
"release_interval": 7200,
"pulse_size": 20
}
},
"status_modifiers": {},
"time_varying_info": {},
"trajectory_modifiers": {},
"velocity_modifiers": {
"my_fall_velocity": {
"class_name": "oceantracker.velocity_modifiers.terminal_velocity.TerminalVelocity",
"mean": -0.001
}
}
}
# now run oceantracker
# as helper "ot" has set params above, simply run it
case_info_file_name, has_errors = ot.run()
# output now in folder "root_output_dir"/"output_file_base"
# in this example output is in directory output/param_test1'
# case_info_file_name the name
# a json file with useful for post processing,
# eg holds output file names to assist in reading data
print('case file name=',case_info_file_name)
startup: -------------------------------------------------------------------------- startup: OceanTracker- preliminary setup startup: Python version: 3.10.10 | packaged by Anaconda, Inc. | (main, Mar 21 2023, 18:39:17) [MSC v.1916 64 bit (AMD64)] startup: - found hydro-model files of type SCHISIM startup: - sorted hyrdo-model files in time order, 0.585 sec startup: >>> Note: output is in dir= e:OneDrive - CawthronH_Local_driveParticleTrackingoceantrackertutorials_how_tooutputparam_test1 startup: >>> Note: to help with debugging, parameters as given by user are in "param_test1_raw_user_params.json" P000: -------------------------------------------------------------------------- P000: Starting case number 0, param_test1 at 2023-06-10T08:47:33.245069 P000: -------------------------------------------------------------------------- P000: - built node to triangles map, 0.634 sec P000: - built triangle adjacency matrix, 0.304 sec P000: - found boundary triangles, 0.000 sec P000: - built domain and island outlines, 1.392 sec P000: - calculated triangle areas, 0.000 sec P000: Finished grid setup P000: - set up release_groups, 0.960 sec P000: - built barycentric-transform matrix, 0.399 sec P000: - initial set up of core classes, 0.403 sec P000: - final set up of core classes, 0.002 sec P000: - created particle properties derived from fields, 0.000 sec P000: >>> Warning: When using a terminal velocity, ensure time step is small enough that vertical displacement is a small fraction of the water depth, ie vertical Courant number < 1 P000: >>> Note: No open boundaries requested, as run_params["open_boundary_type"] = 0 P000: Hint: Requires list of open boundary nodes not in hydro model, eg for Schism this can be read from hgrid file to named in reader params and run_params["open_boundary_type"] = 1 P000: -------------------------------------------------------------------------- P000: - Starting param_test1, duration: 0 days 23 hrs 0 min 0 sec P000: - Reading-file-00 demoHindcastSchism3D.nc, steps in file 24, steps available 000:023, reading 24 of 48 steps, for hydo-model time steps 00:23, from file offsets 00:23, into ring buffer offsets 000:023 P000: - read 24 time steps in 0.4 sec P000: - opening tracks output to : param_test1_tracks.nc P000: 00% step 0000:H0000b00-01 Day +00 00:00 2017-01-01 00:30:00: Rel.:000040: Active:00040 M:00037 S:00000 B:00003 D:000 O:00 N:0 Buffer: 40- 6% step time = 8204.8 ms P000: 04% step 0030:H0001b01-02 Day +00 01:00 2017-01-01 01:30:00: Rel.:000060: Active:00060 M:00054 S:00000 B:00006 D:000 O:00 N:0 Buffer: 60- 8% step time = 2.7 ms P000: 09% step 0060:H0002b02-03 Day +00 02:00 2017-01-01 02:30:00: Rel.:000100: Active:00100 M:00082 S:00001 B:00017 D:000 O:00 N:0 Buffer: 100- 14% step time = 3.1 ms P000: 13% step 0090:H0003b03-04 Day +00 03:00 2017-01-01 03:30:00: Rel.:000120: Active:00120 M:00093 S:00013 B:00014 D:000 O:00 N:0 Buffer: 120- 17% step time = 2.8 ms P000: 17% step 0120:H0004b04-05 Day +00 04:00 2017-01-01 04:30:00: Rel.:000160: Active:00160 M:00136 S:00014 B:00010 D:000 O:00 N:0 Buffer: 160- 22% step time = 3.6 ms P000: 22% step 0150:H0005b05-06 Day +00 05:00 2017-01-01 05:30:00: Rel.:000180: Active:00180 M:00145 S:00016 B:00019 D:000 O:00 N:0 Buffer: 180- 25% step time = 2.9 ms P000: 26% step 0180:H0006b06-07 Day +00 06:00 2017-01-01 06:30:00: Rel.:000220: Active:00220 M:00179 S:00016 B:00025 D:000 O:00 N:0 Buffer: 220- 31% step time = 3.3 ms P000: 30% step 0210:H0007b07-08 Day +00 07:00 2017-01-01 07:30:00: Rel.:000240: Active:00240 M:00201 S:00014 B:00025 D:000 O:00 N:0 Buffer: 240- 33% step time = 2.9 ms P000: 35% step 0240:H0008b08-09 Day +00 08:00 2017-01-01 08:30:00: Rel.:000280: Active:00280 M:00230 S:00013 B:00037 D:000 O:00 N:0 Buffer: 280- 39% step time = 3.8 ms P000: 39% step 0270:H0009b09-10 Day +00 09:00 2017-01-01 09:30:00: Rel.:000300: Active:00300 M:00230 S:00001 B:00069 D:000 O:00 N:0 Buffer: 300- 42% step time = 3.0 ms P000: 43% step 0300:H0010b10-11 Day +00 10:00 2017-01-01 10:30:00: Rel.:000340: Active:00340 M:00269 S:00000 B:00071 D:000 O:00 N:0 Buffer: 340- 47% step time = 3.3 ms P000: 48% step 0330:H0011b11-12 Day +00 11:00 2017-01-01 11:30:00: Rel.:000360: Active:00360 M:00247 S:00000 B:00113 D:000 O:00 N:0 Buffer: 360- 50% step time = 2.9 ms P000: 52% step 0360:H0012b12-13 Day +00 12:00 2017-01-01 12:30:00: Rel.:000400: Active:00400 M:00248 S:00000 B:00152 D:000 O:00 N:0 Buffer: 400- 55% step time = 3.7 ms P000: 57% step 0390:H0013b13-14 Day +00 13:00 2017-01-01 13:30:00: Rel.:000420: Active:00420 M:00286 S:00006 B:00128 D:000 O:00 N:0 Buffer: 420- 58% step time = 3.1 ms P000: 61% step 0420:H0014b14-15 Day +00 14:00 2017-01-01 14:30:00: Rel.:000460: Active:00460 M:00314 S:00010 B:00136 D:000 O:00 N:0 Buffer: 460- 64% step time = 3.5 ms P000: 65% step 0450:H0015b15-16 Day +00 15:00 2017-01-01 15:30:00: Rel.:000480: Active:00480 M:00286 S:00055 B:00139 D:000 O:00 N:0 Buffer: 480- 67% step time = 3.2 ms P000: 70% step 0480:H0016b16-17 Day +00 16:00 2017-01-01 16:30:00: Rel.:000520: Active:00520 M:00294 S:00061 B:00165 D:000 O:00 N:0 Buffer: 520- 72% step time = 3.9 ms P000: 74% step 0510:H0017b17-18 Day +00 17:00 2017-01-01 17:30:00: Rel.:000540: Active:00540 M:00244 S:00071 B:00225 D:000 O:00 N:0 Buffer: 540- 75% step time = 3.1 ms P000: 78% step 0540:H0018b18-19 Day +00 18:00 2017-01-01 18:30:00: Rel.:000580: Active:00580 M:00340 S:00071 B:00169 D:000 O:00 N:0 Buffer: 580- 80% step time = 3.6 ms P000: 83% step 0570:H0019b19-20 Day +00 19:00 2017-01-01 19:30:00: Rel.:000600: Active:00600 M:00357 S:00070 B:00173 D:000 O:00 N:0 Buffer: 600- 83% step time = 3.2 ms P000: 87% step 0600:H0020b20-21 Day +00 20:00 2017-01-01 20:30:00: Rel.:000640: Active:00640 M:00405 S:00064 B:00171 D:000 O:00 N:0 Buffer: 640- 89% step time = 4.2 ms P000: 91% step 0630:H0021b21-22 Day +00 21:00 2017-01-01 21:30:00: Rel.:000660: Active:00660 M:00416 S:00010 B:00234 D:000 O:00 N:0 Buffer: 660- 92% step time = 3.3 ms P000: 96% step 0660:H0022b22-23 Day +00 22:00 2017-01-01 22:30:00: Rel.:000700: Active:00700 M:00389 S:00006 B:00305 D:000 O:00 N:0 Buffer: 700- 97% step time = 3.6 ms P000: 100% step 0689:H0022b22-23 Day +00 22:58 2017-01-01 23:28:00: Rel.:000700: Active:00700 M:00499 S:00000 B:00201 D:000 O:00 N:0 Buffer: 700- 97% step time = 3.9 ms P000: >>> Warning: When using a terminal velocity, ensure time step is small enough that vertical displacement is a small fraction of the water depth, ie vertical Courant number < 1 P000: >>> Note: No open boundaries requested, as run_params["open_boundary_type"] = 0 P000: Hint: Requires list of open boundary nodes not in hydro model, eg for Schism this can be read from hgrid file to named in reader params and run_params["open_boundary_type"] = 1 P000: -------------------------------------------------------------------------- P000: - Finished case number 0, param_test1 started: 2023-06-10 08:47:33.245069, ended: 2023-06-10 08:47:47.833968 P000: Elapsed time =0:00:14.588899 P000: -------------------------------------------------------------------------- P000: - Triangle walk summary: Of 1,007,136 particles located 0, walks were too long and were retried, of these 0 failed after retrying and were discarded startup: >>> Note: run summary with case file names "param_test1_runInfo.json" case file name= e:OneDrive - CawthronH_Local_driveParticleTrackingoceantrackertutorials_how_tooutputparam_test1param_test1_caseInfo.json
Basic plots of tracks¶
also see … for more on plotting notebook
# plot animation of results
from matplotlib import pyplot as plt
from oceantracker.post_processing.plotting.plot_tracks import animate_particles
from oceantracker.post_processing.read_output_files import load_output_files
from IPython.display import HTML # show animation in note book
# read particle track data into a dictionary using case_info_file_name
tracks = load_output_files.load_particle_track_vars(case_info_file_name)
ax= [1591000, 1601500, 5478500, 5491000] # area to plot
# animate particles
anim = animate_particles(tracks, axis_lims=ax,title='Fall vel.+ re-sus., grey part. are on bottom when flows too weak to resuspend',
show_dry_cells=True, show_grid=True, show=False) # use ipython to show video, rather than matplotlib plt.show()
# this is slow to build!
HTML(anim.to_html5_video())
