satpy.readers.yaml_reader module

Base classes and utilities for all readers configured by YAML files.

class satpy.readers.yaml_reader.AbstractYAMLReader(config_dict)

Bases: object

Base class for all readers that use YAML configuration files.

This class should only be used in rare cases. Its child class FileYAMLReader should be used in most cases.

Load information from YAML configuration file about how to read data files.

property all_dataset_ids

Get DataIDs of all datasets known to this reader.

property all_dataset_names

Get names of all datasets known to this reader.

property available_dataset_ids

Get DataIDs that are loadable by this reader.

property available_dataset_names

Get names of datasets that are loadable by this reader.

abstract property end_time

End time of the reader.

abstract filter_selected_filenames(filenames)

Filter provided filenames by parameters in reader configuration.

Returns: iterable of usable files

classmethod from_config_files(*config_files, **reader_kwargs)

Create a reader instance from one or more YAML configuration files.

get_dataset_key(key, **kwargs)

Get the fully qualified DataID matching key.

See satpy.readers.get_key for more information about kwargs.

abstract load(dataset_keys)

Load dataset_keys.

load_ds_ids_from_config()

Get the dataset ids from the config.

select_files_from_directory(directory=None, fs=None)

Find files for this reader in directory.

If directory is None or ‘’, look in the current directory.

Searches the local file system by default. Can search on a remote filesystem by passing an instance of a suitable implementation of fsspec.spec.AbstractFileSystem.

Parameters

• directory (Optional[str]) – Path to search.

• fs (Optional[FileSystem]) – fsspec FileSystem implementation to use. Defaults to None, using local file system.

Returns

list of strings describing matching files

select_files_from_pathnames(filenames)

Select the files from filenames this reader can handle.

property sensor_names

Names of sensors whose data is being loaded by this reader.

abstract property start_time

Start time of the reader.

supports_sensor(sensor)

Check if sensor is supported.

Returns True is sensor is None.

class satpy.readers.yaml_reader.FileYAMLReader(config_dict, filter_parameters=None, filter_filenames=True, **kwargs)

Bases: AbstractYAMLReader, DataDownloadMixin

Primary reader base class that is configured by a YAML file.

This class uses the idea of per-file “file handler” objects to read file contents and determine what is available in the file. This differs from the base AbstractYAMLReader which does not depend on individual file handler objects. In almost all cases this class should be used over its base class and can be used as a reader by itself and requires no subclassing.

Set up initial internal storage for loading file data.

property available_dataset_ids

Get DataIDs that are loadable by this reader.

static check_file_covers_area(file_handler, check_area)

Check if the file covers the current area.

If the file doesn’t provide any bounding box information or ‘area’ was not provided in filter_parameters, the check returns True.

create_filehandlers(filenames, fh_kwargs=None)

Organize the filenames into file types and create file handlers.

property end_time

End time of the latest file used by this reader.

static filename_items_for_filetype(filenames, filetype_info)

Iterate over the filenames matching filetype_info.

filter_fh_by_metadata(filehandlers)

Filter out filehandlers using provide filter parameters.

filter_filenames_by_info(filename_items)

Filter out file using metadata from the filenames.

Currently only uses start and end time. If only start time is available from the filename, keep all the filename that have a start time before the requested end time.

filter_selected_filenames(filenames)

Filter provided files based on metadata in the filename.

find_required_filehandlers(requirements, filename_info)

Find the necessary file handlers for the given requirements.

We assume here requirements are available.

Raises

• KeyError, if no handler for the given requirements is available. –

• RuntimeError, if there is a handler for the given requirements, –

• but it doesn't match the filename info. –

get_dataset_key(key, available_only=False, **kwargs)

Get the fully qualified DataID matching key.

This will first search through available DataIDs, datasets that should be possible to load, and fallback to “known” datasets, those that are configured but aren’t loadable from the provided files. Providing available_only=True will stop this fallback behavior and raise a KeyError exception if no available dataset is found.

Parameters

• key (str, float, DataID, DataQuery) – Key to search for in this reader.

• available_only (bool) – Search only loadable datasets for the provided key. Loadable datasets are always searched first, but if available_only=False (default) then all known datasets will be searched.

• kwargs – See satpy.readers.get_key() for more information about kwargs.

Returns

Best matching DataID to the provided key.

Raises

KeyError – if no key match is found.

load(dataset_keys, previous_datasets=None, **kwargs)

Load dataset_keys.

If previous_datasets is provided, do not reload those.

metadata_matches(sample_dict, file_handler=None)

Check that file metadata matches filter_parameters of this reader.

property sensor_names

Names of sensors whose data is being loaded by this reader.

sorted_filetype_items()

Sort the instance’s filetypes in using order.

property start_time

Start time of the earlier file used by this reader.

time_matches(fstart, fend)

Check that a file’s start and end time mtach filter_parameters of this reader.

update_ds_ids_from_file_handlers()

Add or modify available dataset information.

Each file handler is consulted on whether or not it can load the dataset with the provided information dictionary. See satpy.readers.file_handlers.BaseFileHandler.available_datasets() for more information.

class satpy.readers.yaml_reader.GEOFlippableFileYAMLReader(config_dict, filter_parameters=None, filter_filenames=True, **kwargs)

Bases: FileYAMLReader

Reader for flippable geostationary data.

Set up initial internal storage for loading file data.

class satpy.readers.yaml_reader.GEOSegmentYAMLReader(config_dict, filter_parameters=None, filter_filenames=True, **kwargs)

Bases: GEOFlippableFileYAMLReader

Reader for segmented geostationary data.

This reader pads the data to full geostationary disk if necessary.

This reader uses an optional pad_data keyword argument that can be passed to Scene.load() to control if padding is done (True by default). Passing pad_data=False will return data unpadded.

When using this class in a reader’s YAML configuration, segmented file types (files that may have multiple segments) should specify an extra expected_segments piece of file_type metadata. This tells this reader how many total segments it should expect when padding data. Alternatively, the file patterns for a file type can include a total_segments field which will be used if expected_segments is not defined. This will default to 1 segment.

Set up initial internal storage for loading file data.

create_filehandlers(filenames, fh_kwargs=None)

Create file handler objects and determine expected segments for each.

class satpy.readers.yaml_reader.GEOVariableSegmentYAMLReader(config_dict, filter_parameters=None, filter_filenames=True, **kwargs)

Bases: GEOSegmentYAMLReader

GEOVariableSegmentYAMLReader for handling segmented GEO products with segments of variable height.

This YAMLReader overrides parts of the GEOSegmentYAMLReader to account for formats where the segments can have variable heights. It computes the sizes of the padded segments using the information available in the file(handlers), so that gaps of any size can be filled as needed.

This implementation was motivated by the FCI L1c format, where the segments (called chunks in the FCI world) can have variable heights. It is however generic, so that any future reader can use it. The requirement for the reader is to have a method called get_segment_position_info, returning a dictionary containing the positioning info for each segment (see example in satpy.readers.fci_l1c_nc.FCIL1cNCFileHandler.get_segment_position_info()).

For more information on please see the documentation of satpy.readers.yaml_reader.GEOSegmentYAMLReader().

Initialise the GEOVariableSegmentYAMLReader object.

satpy.readers.yaml_reader.listify_string(something)

Take something and make it a list.

something is either a list of strings or a string, in which case the function returns a list containing the string. If something is None, an empty list is returned.

satpy.readers.yaml_reader.load_yaml_configs(*config_files, loader=<class 'yaml.cyaml.CLoader'>)

Merge a series of YAML reader configuration files.

Parameters

• *config_files (str) – One or more pathnames to YAML-based reader configuration files that will be merged to create a single configuration.

• loader – Yaml loader object to load the YAML with. Defaults to CLoader if libyaml is available, Loader otherwise.

Returns: dict

Dictionary representing the entire YAML configuration with the addition of config[‘reader’][‘config_files’] (the list of YAML pathnames that were merged).

satpy.readers.yaml_reader.split_integer_in_most_equal_parts(x, n)

Split an integer number x in n parts that are as equally-sizes as possible.