Running with helper class¶
[This note-book is in oceantracker/tutorials_how_to/]
Oceantracker is designed so that both coders and non-coders can get almost the same level of adaptability to their specific needs. Also to streamline running many cases across distributed computers, along with enabling online particle tracking as a service.
To achieve these capabilities, it uses parameters to both change the settings to meet the users needs, but also to build the computational pipeline for the users specific needs. This flexibility in the computational pipeline is achieved using “class” parameters.
Classes perform specific roles in the pipeline. Parameters tell Oceantracker which version of core classes the user wants to use for each role, or add optional classes. Eg. add multiple “release_groups” classes, which may release particles at points, or within a polygon, at different times, rates and locations, all within the same computational run.
Simply by changing the parameters, users can load their own version of a class, including the “solver” class which orchestrates the time stepping and operations by other classes. There are base classes for all the major roles within the computational pipeline. Classes can be adapted by inheriting the bases class, or one of its children, and overwriting some of the methods to alter how the class performs its role.
To make the ideas of settings and classes easier to adopt, the below uses methods of a helper class to add settings and classes to the pipeline. The notebook E_run_using_parameter_dictionaries.ipynb, shows how run directly from parameter dictionaries, which are built in code or read from a json or yaml file.
Parameters using helper¶
There are two types of parameters:
Settings parameters
These are set with one or more calls to helper method, ot.setting(…), where … are keyword arguments. eg “time_step”, the model time step in seconds is set using, time_step= 1800.
There are default values for most settings. A full set of settings and their defaults is at all settings
Class role parameters
Classes add specific tasks to the computational pipeline, eg. how to release particles, write output, particle suspension etc. These are added using helper method ot.add_class(class_role, ….). The first argument is a “class_role”. Each class has its own specific settings, which are set using keyword arguments of helper method. Eg. ot.add_class(‘reader’, input_dir= ‘my_hindcast_dir’, ….) adds a reader class and tells is it which folder contains the hindcast files.
A full set of classes and their default settings is at parameter_ref
Normally there must be a keyword argument “class_name” setting. This is used to import the required class into the computational pipeline, with its specific settings. This “class_name” is a string, used to import the class at setup (so is the same as that used to import any python class within a module).
The “class_name” setting is normally required. A few class_roles have a default class_name, eg. in minimal_example “release_groups” assumes a point_release, and the reader’s “class_name” is chosen by looking at variable names within the hindcast’s netcdf file.
There are two types of “class_role”.
When
Only a single class is required, these have singular “class_role”, eg.”reader” and “solver”.
One or more classes can be added. These have plural “class_role” (eg. can add multiple “release_groups”). Users must give a unique name to each one.
Documentation for all parameters/settings and their default values.
Add link here
Extend the minimal example¶
The below extends the minimal_example, it adds:
release at random locations within a polygon, at random water depths (the default)
a particle fall velocity
particles on the bottom are re-suspended if friction velocity exceeds a critical value.
# build parameters using helper class
from oceantracker.main import OceanTracker
ot = OceanTracker() # make an instance of the helper class
# one or more settings can be set by calls to os.settings
ot.settings(output_file_base='param_test1',# name used as base for output files
root_output_dir='output' # output is put in dir 'root_output_dir'/'output_file_base'
)
# ot.settings can be used more than once to add more settings
ot.settings(time_step =120) # 2 min model time step as seconds
# add a compulsory reader class
# no class_name setting is required
# as will detect that it needs a a schism reader class
ot.add_class('reader',input_dir= '../demos/demo_hindcast/schsim3D', # folder to search for hindcast files, sub-dirs will, by default, also be searched
file_mask= 'demo_hindcast_schisim3D*.nc') # hindcast file mask
# add some release groups
# release_groups are one or more release groups
# (ie locations where particles are released at the same times and locations)
# there must be at least one release group
# add release locations from two points,
ot.add_class('release_groups', # class_role is release_groups
name ='my_release_points1',
# the required name setting, is used to refer to this release group internally anf in postprocsing
points= [[1595000, 5482600],
[1599000, 5486200]], # must be an N by 2 or 3 or list, convertible to a numpy array
release_interval= 3600, # seconds between releasing particles
pulse_size = 10, # number of particles released each release_interval
# no class_name setting, so default is a point release
)
# add a second release group, from random locations within a polygon
ot.add_class('release_groups', # class_role is release_group
name ='my_polygon_release1',
class_name= 'PolygonRelease', # use a polygon release
# this time is a polygon , so below points are polygon cords
points = [ [1597682.1237, 5489972.7479],
[1598604.1667, 5490275.5488],
[1598886.4247, 5489464.0424],
[1597917.3387, 5489000],
[1597300, 5489000],
[1597682.1237, 5489972.7479]
],
release_interval= 7200, # seconds between releasing particles
pulse_size = 20, # number of particles released each release_interval
)
# alter default re-suspension class's default settings
ot.add_class('resuspension', critical_friction_velocity = .005) # only re-suspend particles if friction vel. exceeds this value
# add a class to modify the particle velocity
# velocity_modifiers are a set of velocities added to water velocity given in the hydrodynamic model's
# here a fall velocity with a given value is added to the computation
ot.add_class('velocity_modifiers', # class_role is velocity_modifiers
name ='my_fall_velocity',
class_name = 'TerminalVelocity',
value= -0.001, # mean terminal vel < 0 for falling
# optionally variance can also be use to give each particle its own fall velocity
)
# now run oceantracker
# as helper "ot" has set params above, simply run it
case_info_file_name = ot.run()
# output now in folder "root_output_dir"/"output_file_base"
# in this example output is in directory output/param_test1'
# case_info_file_name is the name of a json file with useful for post processing,
# eg it holds output file names to assist in reading and plotting data
print('case file name=',case_info_file_name)
helper ---------------------------------------------------------------------- helper Starting OceanTracker helper class helper - Starting run using helper class Main Python version: 3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:27:10) [MSC v.1938 64 bit (AMD64)] Main >>> Warning: Oceantracker is not yet compatible with Python 3.11, as not all imported packages have been updated, eg netcdf4 Main ---------------------------------------------------------------------- Main OceanTracker starting main: Main Starting package set up Main - Built OceanTracker package tree, 0.573 sec Main - Built OceanTracker sort name map, 0.000 sec Main - Done package set up to setup ClassImporter, 0.574 sec Main >>> Warning: Deleted contents of existing output dir Main Output is in dir "f:H_Local_driveParticleTrackingoceantrackertutorials_how_tooutputparam_test1" Main hint: see for copies of screen output and user supplied parameters, plus all other output Main >>> Note: to help with debugging, parameters as given by user are in "user_given_params.json" Main ---------------------------------------------------------------------- Main OceanTracker version 0.50.0010-2024-03-30 - preliminary setup Main - Found input dir "../demos/demo_hindcast/schsim3D" Main - found hydro-model files of type "SCHISM" Main Cataloging hindcast with 1 files in dir ../demos/demo_hindcast/schsim3D Main - Cataloged hydro-model files/variables in time order, 0.009 sec Main >>> Note: No bottom_stress variable in in hydro-files, using near seabed velocity to calculate friction_velocity for resuspension Main - sorted hyrdo-model files in time order, 0.031 sec prelim: Starting package set up prelim: - Built OceanTracker package tree, 0.009 sec prelim: - Built OceanTracker sort name map, 0.000 sec prelim: - Done package set up to setup ClassImporter, 0.009 sec C000 ---------------------------------------------------------------------- C000 Starting case number 0, param_test1 at 2024-09-04T08:40:49.310527 C000 ---------------------------------------------------------------------- C000 - Scanned OceanTracker to build short name map to the full class_names, 0.000 sec C000 >>> Note: Hydro-model is "3D" type "SCHISMreaderNCDF" C000 hint: Files found dir and sub-dirs of "../demos/demo_hindcast/schsim3D" C000 Start: 2017-01-01T00:30:00.000000000 end: 2017-01-01T23:30:00.000000000, time steps 24 C000 grid bounding box = [1589789.0 5479437.0] to [1603398.0 5501640.0] C000 - Starting grid setup C000 - built node to triangles map, 0.894 sec C000 - built triangle adjacency matrix, 0.156 sec C000 - found boundary triangles, 0.000 sec C000 - built domain and island outlines, 0.963 sec C000 - calculated triangle areas, 0.000 sec C000 - Finished grid setup C000 - built barycentric-transform matrix, 0.221 sec C000 >>> Note: Hydro-model grid in metres, all cords should be in meters, e.g. release group locations, gridded_stats grid C000 - Setup field group manager, 0.223 sec C000 - Added release groups and found run start and end times, 0.003 sec C000 >>> Note: When using a terminal velocity, ensure time step is small enough that vertical displacement is a small fraction of the water depth, ie vertical Courant number < 1 C000 - Done initial setup of all classes, 0.548 sec C000 >>> Note: Hydro-model grid in metres, all cords should be in meters, e.g. release group locations, gridded_stats grid C000 ---------------------------------------------------------------------- C000 - Starting param_test1, duration: 0 days 23 hrs 0 min 0 sec C000 - Reading 24 time steps, for hindcast time steps 00:23, into ring buffer offsets 000:023 C000 - read 24 time steps in 0.9 sec C000 ---------------------------------------------------------------------- C000 - Starting time stepping: 2017-01-01T00:30:00 to 2017-01-01T23:30:00 , duration 0 days 23 hrs 0 min 0 sec C000 - opening tracks output to : param_test1_tracks_compact.nc C000 00% step 0000:H0000b00-01 Day +00 00:00 2017-01-01 00:30:00: Rel.: 58: Active:00058 M:00058 S:00000 B:00000 D:000 O:00 N:000 Buffer:0058 0% step time = 3733.7 ms C000 04% step 0030:H0001b01-02 Day +00 01:00 2017-01-01 01:30:00: Rel.: 78: Active:00078 M:00077 S:00000 B:00001 D:000 O:00 N:000 Buffer:0078 0% step time = 1.7 ms C000 09% step 0060:H0002b02-03 Day +00 02:00 2017-01-01 02:30:00: Rel.: 139: Active:00139 M:00129 S:00000 B:00010 D:000 O:00 N:000 Buffer:0139 0% step time = 1.9 ms C000 13% step 0090:H0003b03-04 Day +00 03:00 2017-01-01 03:30:00: Rel.: 159: Active:00159 M:00146 S:00010 B:00003 D:000 O:00 N:000 Buffer:0159 0% step time = 1.7 ms C000 17% step 0120:H0004b04-05 Day +00 04:00 2017-01-01 04:30:00: Rel.: 204: Active:00204 M:00183 S:00012 B:00009 D:000 O:00 N:000 Buffer:0204 0% step time = 1.8 ms C000 22% step 0150:H0005b05-06 Day +00 05:00 2017-01-01 05:30:00: Rel.: 224: Active:00224 M:00196 S:00018 B:00010 D:000 O:00 N:000 Buffer:0224 0% step time = 1.7 ms C000 26% step 0180:H0006b06-07 Day +00 06:00 2017-01-01 06:30:00: Rel.: 280: Active:00280 M:00247 S:00018 B:00015 D:000 O:00 N:000 Buffer:0280 0% step time = 1.9 ms C000 30% step 0210:H0007b07-08 Day +00 07:00 2017-01-01 07:30:00: Rel.: 300: Active:00300 M:00267 S:00012 B:00021 D:000 O:00 N:000 Buffer:0300 0% step time = 2.7 ms C000 35% step 0240:H0008b08-09 Day +00 08:00 2017-01-01 08:30:00: Rel.: 356: Active:00356 M:00327 S:00010 B:00019 D:000 O:00 N:000 Buffer:0356 0% step time = 2.0 ms C000 39% step 0270:H0009b09-10 Day +00 09:00 2017-01-01 09:30:00: Rel.: 376: Active:00376 M:00350 S:00000 B:00026 D:000 O:00 N:000 Buffer:0376 0% step time = 1.8 ms C000 43% step 0300:H0010b10-11 Day +00 10:00 2017-01-01 10:30:00: Rel.: 429: Active:00429 M:00391 S:00000 B:00038 D:000 O:00 N:000 Buffer:0429 0% step time = 1.9 ms C000 48% step 0330:H0011b11-12 Day +00 11:00 2017-01-01 11:30:00: Rel.: 449: Active:00449 M:00393 S:00000 B:00056 D:000 O:00 N:000 Buffer:0449 0% step time = 1.6 ms C000 52% step 0360:H0012b12-13 Day +00 12:00 2017-01-01 12:30:00: Rel.: 490: Active:00490 M:00425 S:00000 B:00065 D:000 O:00 N:000 Buffer:0490 0% step time = 1.8 ms C000 57% step 0390:H0012b12-13 Day +00 13:00 2017-01-01 13:30:00: Rel.: 510: Active:00510 M:00454 S:00004 B:00052 D:000 O:00 N:000 Buffer:0510 0% step time = 1.8 ms C000 61% step 0420:H0014b14-15 Day +00 14:00 2017-01-01 14:30:00: Rel.: 552: Active:00552 M:00479 S:00005 B:00068 D:000 O:00 N:000 Buffer:0552 0% step time = 2.0 ms C000 65% step 0450:H0015b15-16 Day +00 15:00 2017-01-01 15:30:00: Rel.: 572: Active:00572 M:00437 S:00064 B:00071 D:000 O:00 N:000 Buffer:0572 0% step time = 1.8 ms C000 70% step 0480:H0016b16-17 Day +00 16:00 2017-01-01 16:30:00: Rel.: 631: Active:00631 M:00465 S:00066 B:00100 D:000 O:00 N:000 Buffer:0631 0% step time = 2.0 ms C000 74% step 0510:H0017b17-18 Day +00 17:00 2017-01-01 17:30:00: Rel.: 651: Active:00651 M:00369 S:00070 B:00212 D:000 O:00 N:000 Buffer:0651 0% step time = 1.7 ms C000 78% step 0540:H0018b18-19 Day +00 18:00 2017-01-01 18:30:00: Rel.: 692: Active:00692 M:00423 S:00070 B:00199 D:000 O:00 N:000 Buffer:0692 0% step time = 1.9 ms C000 83% step 0570:H0019b19-20 Day +00 19:00 2017-01-01 19:30:00: Rel.: 712: Active:00712 M:00514 S:00070 B:00128 D:000 O:00 N:000 Buffer:0712 0% step time = 1.8 ms C000 87% step 0600:H0020b20-21 Day +00 20:00 2017-01-01 20:30:00: Rel.: 764: Active:00764 M:00576 S:00068 B:00120 D:000 O:00 N:000 Buffer:0764 0% step time = 2.1 ms C000 91% step 0630:H0021b21-22 Day +00 21:00 2017-01-01 21:30:00: Rel.: 784: Active:00784 M:00662 S:00005 B:00117 D:000 O:00 N:000 Buffer:0784 0% step time = 1.7 ms C000 96% step 0660:H0022b22-23 Day +00 22:00 2017-01-01 22:30:00: Rel.: 825: Active:00825 M:00692 S:00004 B:00129 D:000 O:00 N:000 Buffer:0825 0% step time = 1.9 ms C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.0 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.0 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.0 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.0 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.0 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.0 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.1 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.0 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.0 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.0 sec C000 - Reading 1 time steps, for hindcast time steps 23:23, into ring buffer offsets 023:023 C000 - read 1 time steps in 0.0 sec C000 100% step 0690:H0023b23-00 Day +00 23:00 2017-01-01 23:30:00: Rel.: 825: Active:00825 M:00660 S:00000 B:00165 D:000 O:00 N:000 Buffer:0825 0% step time = 51.5 ms C000 >>> Note: Hydro-model is "3D" type "SCHISMreaderNCDF" C000 hint: Files found dir and sub-dirs of "../demos/demo_hindcast/schsim3D" C000 >>> Note: Hydro-model grid in metres, all cords should be in meters, e.g. release group locations, gridded_stats grid C000 >>> Note: When using a terminal velocity, ensure time step is small enough that vertical displacement is a small fraction of the water depth, ie vertical Courant number < 1 C000 >>> Note: Hydro-model grid in metres, all cords should be in meters, e.g. release group locations, gridded_stats grid C000 ---------------------------------------------------------------------- C000 - Finished case number 0, param_test1 started: 2024-09-04 08:40:49.296859, ended: 2024-09-04 08:41:01.827638 C000 Computational time =0:00:12.530779 C000 --- End case 0 ------------------------------------------------------- End --- Summary ---------------------------------------------------------- End >>> Note: Run summary with case file names in "*_runInfo.json" End >>> Note: to help with debugging, parameters as given by user are in "user_given_params.json" End >>> Note: No bottom_stress variable in in hydro-files, using near seabed velocity to calculate friction_velocity for resuspension End >>> Note: Run summary with case file names in "*_runInfo.json" End >>> Warning: Oceantracker is not yet compatible with Python 3.11, as not all imported packages have been updated, eg netcdf4 End >>> Warning: Deleted contents of existing output dir End ---------------------------------------------------------------------- End ---------------------------------------------------------------------- End OceanTracker summary: elapsed time =0:00:13.168148 End Cases - 0 errors, 0 warnings, 4 notes, check above End Main - 0 errors, 2 warnings, 3 notes, check above End Output in f:H_Local_driveParticleTrackingoceantrackertutorials_how_tooutputparam_test1 End ---------------------------------------------------------------------- case file name= f:H_Local_driveParticleTrackingoceantrackertutorials_how_tooutputparam_test1param_test1_caseInfo.json
# can see the parameters used to build
# using the helper class instance in ot.params
import json
print(json.dumps(ot.params, indent=4))
{
"output_file_base": "param_test1",
"root_output_dir": "output",
"time_step": 120,
"reader": {
"input_dir": "../demos/demo_hindcast/schsim3D",
"file_mask": "demo_hindcast_schisim3D*.nc"
},
"release_groups": [
{
"points": [
[
1595000,
5482600
],
[
1599000,
5486200
]
],
"release_interval": 3600,
"pulse_size": 10,
"name": "my_release_points1"
},
{
"points": [
[
1597682.1237,
5489972.7479
],
[
1598604.1667,
5490275.5488
],
[
1598886.4247,
5489464.0424
],
[
1597917.3387,
5489000
],
[
1597300,
5489000
],
[
1597682.1237,
5489972.7479
]
],
"release_interval": 7200,
"pulse_size": 20,
"class_name": "PolygonRelease",
"name": "my_polygon_release1"
}
],
"resuspension": {
"critical_friction_velocity": 0.005
},
"velocity_modifiers": [
{
"value": -0.001,
"class_name": "TerminalVelocity",
"name": "my_fall_velocity"
}
]
}
Basic plots of tracks¶
# plot animation of results
from matplotlib import pyplot as plt
from plot_oceantracker.plot_tracks import animate_particles
from read_oceantracker.python import load_output_files
from IPython.display import HTML # show animation in note book
# read particle track data into a dictionary using case_info_file_name
tracks = load_output_files.load_track_data(case_info_file_name)
ax= [1591000, 1601500, 5479500, 5491000] # area to plot
# animate particles
anim = animate_particles(tracks, axis_lims=ax,title='Fall vel.+ re-sus., grey part. are on bottom when flows too weak to resuspend',
show_dry_cells=True, show_grid=True, show=False) # use ipython to show video, rather than matplotlib plt.show()
# this is slow to build!
HTML(anim.to_html5_video())
