cedar.stores.gdrive module

Helper utilities for using Google Drive

class cedar.stores.gdrive.GDriveStore(service)[source]

Bases: object

Store GEE “pre-ARD” images and metadata on Google Drive

Parameters

service (googleapiclient.discovery.Resource) – Google Drive API service

classmethod from_credentials(client_secrets_file=None, credentials_file=None)[source]

Create and/or load credentials and create the store

Parameters

  • client_secrets_file (str or Path) – Filename of “client_secrets.[…].json” file

  • credentials_file (str or Path) – Filename of user credentials to load, or to save to for future use. If not provided, will use default location.

list(self, path=None, pattern=None)[source]

List stored images or metadata

Parameters

  • path (str, optional) – Prefix path to search within

  • pattern (str, optional) – Filename pattern

Returns

Names of stored data

Return type

list[str]

read_metadata(self, name, path=None)[source]

Read and parse JSON metadata into a dict

Parameters

  • name (str) – Filename of metadata to read

  • path (str, optional) – Parent directory for file/object stored

Returns

JSON metadata

Return type

dict

remove(self, name, path=None)[source]

Remove a file from Google Drive

Parameters

  • name (str) – Name of stored file/object

  • path (str, optional) – Parent directory for file/object stored

Returns

Name of file removed

Return type

str

retrieve_image(self, dest, name, path=None, overwrite=True)[source]

Retrieve (pieces of) an image from the Google Drive

Parameters

  • dest (str) – Destination folder to save image(s)

  • name (str) – Name of stored file/object

  • path (str, optional) – Parent directory for file/object stored on Google Drive

Yields

Sequence[str] – Filename(s) corresponding to retrieved data

retrieve_metadata(self, dest, name, path=None, overwrite=True)[source]

Retrieve image metadata from the GCS

Parameters

  • dest (str) – Destination folder to save metadata

  • name (str) – Name of stored file/object

  • path (str, optional) – Parent directory for file/object stored

Yields

pathlib.Path – Filename corresponding to retrieved data

store_image(self, image, name, path=None, **export_image_kwds)[source]

Create ee.batch.Task to create and store “pre-ARD”

Parameters

  • image (ee.Image) – Earth Engine image to compute & store

  • name (str) – Name of file/object to store

  • path (str, optional) – Parent directory for file/object stored

  • export_image_kwds (dict, optional) – Additional keyword arguments to pass onto ee.batch.Export.image.toCloudStorage() (hint: scale & crs)

Returns

Earth Engine Task

Return type

ee.Task

store_metadata(self, metadata, name, path=None)[source]

Store JSON metadata

Parameters

  • metadata (dict) – Metadata, to be saved as JSON

  • name (str) – Name of file/object to store

  • path (str, optional) – Parent directory for file/object stored

Returns

ID of file uploaded

Return type

str

cedar.stores.gdrive.build_gdrive_service(credentials)[source]

Return Google Drive API service, either by credentials or file

Parameters

credentials (google.oauth2.credentials.Credentials) – User credentials, including access token, for using the application.

Returns

GDrive v3 API resource

Return type

googleapiclient.discovery.Resource

cedar.stores.gdrive.delete(service, name, parent_id=None, appProperties=False)[source]

Delete a file/folder on Google Drive

Parameters

  • service (googleapiclient.discovery.Resource) – Google API resource for GDrive v3

  • name (str) – Name of file/folder

  • parent_id (str, optional) – Parent ID of folder containing file (to narrow search)

  • appProperties (bool) – Search for application-specific files using appProperties

Returns

ID of deleted file/folder

Return type

str

cedar.stores.gdrive.download_file(service, name, dest, parent_id=None)[source]

Download a file to a destination directory

Parameters

  • service (googleapiclient.discovery.Resource) – Google API resource for GDrive v3

  • dest (str) – Name of destination

  • parent_id (str, optional) – Parent ID of folder containing file (to narrow search)

Returns

Filename written to

Return type

pathlib.Path

Raises

  • FileExistsError – Raised if file exists in destination but not allowed to overwrite,

  • ValueError – Raised if the file given does not exist in Google Drive

cedar.stores.gdrive.download_file_id(service, file_id, dest)[source]

Download a file to a destination directory using its ID

Parameters

  • service (googleapiclient.discovery.Resource) – Google API resource for GDrive v3

  • file_id (str) – ID of file on Google Drive

  • dest (str or pathlib.Path) – Destination filename

Returns

Filename written to

Return type

pathlib.Path

Raises

  • FileExistsError – Raised if file exists in destination but not allowed to overwrite,

  • ValueError – Raised if the file given does not exist in Google Drive

cedar.stores.gdrive.exists(service, name, parent_id=None, directory=False, trashed=False, appProperties=False)[source]

Check if file/folder exists

Parameters

  • service (googleapiclient.discovery.Resource) – Google API resource for GDrive v3

  • name (str) – Name of file/folder

  • parent_id (str, optional) – Parent ID of folder containing file (to narrow search)

  • directory (bool, optional) – True if file needs to also be a directory

  • trashed (bool, optional) – Search in the trash?

  • appProperties (bool) – Search for application-specific files using appProperties

Returns

Returns object ID if exists, otherwise empty string

Return type

str

cedar.stores.gdrive.find_credentials(client_secrets_file=None, credentials_file=None)[source]

Locate GDrive client secrets & credentials files

cedar.stores.gdrive.get_appProperties()[source]

Returns private “appProperties” to include when creating files

cedar.stores.gdrive.get_credentials(client_secrets_file=None, credentials_file=None, no_browser=True)[source]

Get OAuth2 Credentials for Google Drive

Parameters

  • client_secrets_file (str or Path) – Filename of “client_secrets.[…].json” file

  • credentials_file (str or Path) – Filename of user credentials to load, or to save to for future use. If not provided, will use default location.

  • no_browser (bool, optional) – Disables opening a web browser in favor of prompting user to authenticate using a terminal prompt.

Returns

  • credentials (google.oauth2.credentials.Credentials) – User credentials, including access token, for using the application.

  • credentials_file (str) – File storing user credentials

cedar.stores.gdrive.list_dirs(service, parent_id=None, name=None)[source]

List folders on Google Drive

Parameters

  • service (googleapiclient.discovery.Resource) – Google API resource for GDrive v3

  • parent_id (str, optional) – Parent ID of folder to list (to narrow search)

  • name (str, optional) – Name to search for (don’t include asterisks)

Yields

list[dict] – Info about objects stored on Google Drive (keys=(‘id’, ‘name’, ))

cedar.stores.gdrive.list_objects(service, parent_id=None, name=None, q=None, appProperties=False)[source]

List files/folders on Google Drive

Parameters

  • service (googleapiclient.discovery.Resource) – Google API resource for GDrive v3

  • parent_id (str, optional) – Parent ID of folder to list (to narrow search)

  • name (str, optional) – Name to search for (don’t include asterisks)

  • q (str or Sequence[str], optional) – Additional search query parameters

  • appProperties (bool) – Search for application-specific files using appProperties

Yields

list[dict] – Info about objects stored on Google Drive (keys=(‘id’, ‘name’, ))

cedar.stores.gdrive.load_credentials(filename)[source]

Load Google OAuth2 credentials from file

Returns

Return type

google.oauth2.credentials.Credentials

cedar.stores.gdrive.mkdir(service, name, parent_id=None, check=False)[source]

Make a directory on GDrive

Parameters

  • service (googleapiclient.discovery.Resource) – Google API resource for GDrive v3

  • name (str) – Directory to create

  • parent_id (str, optional) – ID of parent directory

  • check (bool, optional) – Check if directory exists before creating. If exists, will not create a new directory and instead return the ID of this directory

Returns

Google Drive ID for directory created (or already existing)

Return type

str

cedar.stores.gdrive.mkdir_p(service, dest, parent_id=None)[source]

Make a directory, recursively

Parameters

  • service (googleapiclient.discovery.Resource) – Google API resource for GDrive v3

  • dest (str) – Directory to create

Returns

Google Drive ID for directory created (or already existing)

Return type

str

cedar.stores.gdrive.query_appProperties()[source]

Build a query component for appProperties

cedar.stores.gdrive.read_json(service, name, parent_id=None)[source]

Reads and returns a JSON file from Google Drive

Parameters

  • service (googleapiclient.discovery.Resource) – Google API resource for GDrive v3

  • name (str) – Name of file/folder

  • parent_id (str, optional) – Parent ID of folder containing file (to narrow search)

Returns

JSON data as a dict

Return type

dict

cedar.stores.gdrive.save_credentials(credentials, filename)[source]

Save Google OAuth2 credentials

Parameters

  • credentials (google.oauth2.credentials.Credentials) – Google OAuth2 credentials

  • filename (str or Path) – Path to save credentials

cedar.stores.gdrive.upload_json(service, data, name, path=None, check=False, encoding='utf-8')[source]

Upload JSON data to Google Drive

Parameters

  • service (googleapiclient.discovery.Resource) – Google API resource for GDrive v3

  • data (str or dict) – JSON, either already dumped to str or as a dict

  • name (str) – Filename for data

  • path (str, optional) – Parent directory to put file. If None, stores in root of Google Drive

  • check (bool, optional) – Check to see if the file already exists first. If so, will overwrite (or “update” instead of “create”)

  • encoding (str, optional) – Metadata encoding

Returns

ID of file uploaded

Return type

str