Skip to content

Parquet

Description of Parquet files

describe_file 

describe_file(fn: str | Path, bucket: Bucket | None = None) -> list[dict]

Describe objects (always a single object) in an individual parquet file.

unify_descriptions 

unify_descriptions(descriptions: Collection[FileDescription]) -> tuple[list[dict] | None, str | None]

Attempt to unify a collection of Parquet file descriptions into a list of dicts suitable for use as the DataObjects of a Filetype.

Parameters:

  • descriptions (Collection[FileDescription]) –

    collection of FileDescriptions populated with describe_file()

Returns:

  • objects ( list[dict] | None ) –

    if unification succeeded, a list of dicts that can be used to initialize DataObjects; if it didn't, None

  • failure ( str | None ) –

    if unification failed, a string describing the failure; if it succeeded, None