satpy.aux_download module
Functions and utilities for downloading ancillary data.
- class satpy.aux_download.DataDownloadMixin[source]
Bases:
object
Mixin class for Satpy components to download files.
This class simplifies the logic needed to download and cache data files needed for operations in a Satpy component (readers, writers, etc). It does this in a two step process where files that might be downloaded are “registered” and then “retrieved” when they need to be used.
To use this class include it as one of the subclasses of your Satpy component. Then in the
__init__
method, call theregister_data_files
function during initialization.Note
This class is already included in the
FileYAMLReader
andWriter
base classes. There is no need to define a custom class.The below code is shown as an example:
from satpy.readers.yaml_reader import AbstractYAMLReader from satpy.aux_download import DataDownloadMixin class MyReader(AbstractYAMLReader, DataDownloadMixin): def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self.register_data_files()
This class expects data files to be configured in either a
self.info['data_files']
(standard for readers/writers) orself.config['data_files']
list. Thedata_files
item itself is a list of dictionaries. This information can also be passed directly toregister_data_files
for more complex cases. In YAML, for a reader, this might look like this:reader: name: abi_l1b short_name: ABI L1b long_name: GOES-R ABI Level 1b ... other metadata ... data_files: - url: "https://example.com/my_data_file.dat" - url: "https://raw.githubusercontent.com/pytroll/satpy/main/README.rst" known_hash: "sha256:5891286b63e7745de08c4b0ac204ad44cfdb9ab770309debaba90308305fa759" - url: "https://raw.githubusercontent.com/pytroll/satpy/main/RELEASING.md" filename: "satpy_releasing.md"
In this example we register two files that might be downloaded. If
known_hash
is not provided or None (null in YAML) then the data file will not be checked for validity when downloaded. Seeregister_file()
for more information. You can optionally specifyfilename
to define the in-cache name when this file is downloaded. This can be useful in cases when the filename can not be easily determined from the URL.When it comes time to needing the file, you can retrieve the local path by calling
~satpy.aux_download.retrieve(cache_key)
with the “cache key” generated during registration. These keys will be in the format:<component_type>/<filename>
. For a reader this would bereaders/satpy_release.md
.This Mixin is not the only way to register and download files for a Satpy component, but is the most generic and flexible. Feel free to use the
register_file()
andretrieve()
functions directly. However,find_registerable_files()
must also be updated to support your component (if files are not register during initialization).- DATA_FILE_COMPONENTS = {'composit': 'composites', 'corr': 'modifiers', 'modifi': 'modifiers', 'reader': 'readers', 'writer': 'writers'}
- register_data_files(data_files=None)[source]
Register a series of files that may be downloaded later.
See
DataDownloadMixin
for more information on the assumptions and structure of the data file configuration dictionary.
- satpy.aux_download.find_registerable_files(readers=None, writers=None, composite_sensors=None)[source]
Load all Satpy components so they can be downloaded.
- Parameters:
readers (list or None) – Limit searching to these readers. If not specified or
None
then all readers are searched. If an empty list then no readers are searched.writers (list or None) – Limit searching to these writers. If not specified or
None
then all writers are searched. If an empty list then no writers are searched.composite_sensors (list or None) – Limit searching to composite configuration files for these sensors. If
None
then all sensor configs will be searched. If an empty list then no composites will be searched.
- satpy.aux_download.register_file(url, filename, component_type=None, known_hash=None)[source]
Register file for future retrieval.
This function only prepares Satpy to be able to download and cache the provided file. It will not download the file. See
satpy.aux_download.retrieve()
for more information.- Parameters:
url (str) – URL where remote file can be downloaded.
filename (str) – Filename used to identify and store the downloaded file as.
component_type (str or None) – Name of the type of Satpy component that will use this file. Typically “readers”, “composites”, “writers”, or “enhancements” for consistency. This will be prepended to the filename when storing the data in the cache.
known_hash (str) – Hash used to verify the file is downloaded correctly. See https://www.fatiando.org/pooch/v1.3.0/beginner.html#hashes for more information. If not provided then the file is not checked.
- Returns:
Cache key that can be used to retrieve the file later. The cache key consists of the
component_type
and providedfilename
. This should be passed tosatpy.aux_download_retrieve()
when the file will be used.
- satpy.aux_download.retrieve(cache_key, pooch_kwargs=None)[source]
Download and cache the file associated with the provided
cache_key
.Cache location is controlled by the config
data_dir
key. See Data Directory for more information.- Parameters:
cache_key (str) – Cache key returned by
register_file()
.pooch_kwargs (dict or None) – Extra keyword arguments to pass to
pooch.Pooch.fetch()
.
- Returns:
Local path of the cached file.
- satpy.aux_download.retrieve_all(readers=None, writers=None, composite_sensors=None, pooch_kwargs=None)[source]
Find cache-able data files for Satpy and download them.
The typical use case for this function is to download all ancillary files before going to an environment/system that does not have internet access.
- Parameters:
readers (list or None) – Limit searching to these readers. If not specified or
None
then all readers are searched. If an empty list then no readers are searched.writers (list or None) – Limit searching to these writers. If not specified or
None
then all writers are searched. If an empty list then no writers are searched.composite_sensors (list or None) – Limit searching to composite configuration files for these sensors. If
None
then all sensor configs will be searched. If an empty list then no composites will be searched.pooch_kwargs (dict) – Additional keyword arguments to pass to pooch
fetch
.