satpy.writers.core.compute module

Utilities for writers.

satpy.writers.core.compute.compute_writer_results(results)

Compute all the given dask graphs results so that the files are saved.

Parameters:

results (Iterable) – Iterable of dask graphs resulting from calls to scn.save_datasets(…, compute=False)

satpy.writers.core.compute.group_results_by_output_file(sources, targets)

Group results by output file.

For writers that return sources and targets for compute=False, split the results by output file.

When not only the data but also GeoTIFF tags are dask arrays, then save_datasets(..., compute=False)` returns a tuple of flat lists, where the second list consists of a mixture of RIOTag and RIODataset objects (from trollimage). In some cases, we may want to get a seperate delayed object for each file; for example, if we want to add a wrapper to do something with the file as soon as it’s finished. This function unflattens the flat lists into a list of (src, target) tuples.

For example, to close files as soon as computation is completed:

>>> @dask.delayed
>>> def closer(obj, targs):
...     for targ in targs:
...         targ.close()
...     return obj
>>> (srcs, targs) = sc.save_datasets(writer="ninjogeotiff", compute=False, **ninjo_tags)
>>> for (src, targ) in group_results_by_output_file(srcs, targs):
...     delayed_store = da.store(src, targ, compute=False)
...     wrapped_store = closer(delayed_store, targ)
...     wrapped.append(wrapped_store)
>>> compute_writer_results(wrapped)

In the wrapper you can do other useful tasks, such as writing a log message or moving files to a different directory.

Warning

Adding a callback may impact runtime and RAM. The pattern or cause is unclear. Tests with FCI data show that for resampling with high RAM use (from around 15 GB), runtime increases when a callback is added. Tests with ABI or low RAM consumption rather show a decrease in runtime. More information, see these GitHub comments Users who find out more are encouraged to contact the Satpy developers with clues.

Parameters:

• sources – List of sources (typically dask.array) as returned by satpy.scene.Scene.save_datasets().

• targets – List of targets (should be RIODataset or RIOTag) as returned by satpy.scene.Scene.save_datasets().

Returns:

List of Tuple(List[sources], List[targets]) with a length equal to the number of output files planned to be written by satpy.scene.Scene.save_datasets().

satpy.writers.core.compute.split_results(results)

Split results.

Get sources, targets and delayed objects to separate lists from a list of results collected from (multiple) writer(s).