modules.dataClean

Null-value checking for TableData, TableCollection, and ResultModel.

Classes

class modules.dataClean.NullValueChecker

Check specified columns for null values and react per column policy.

This module does not modify data. The input object is passed through unchanged. Null values are None, NaN, NaT, and pd.NA. Empty strings "" are also treated as null when treat_empty_string_as_null is True. Empty lists [] and empty tuples () are treated as null when treat_empty_list_as_null is True.

When a column or list field is entirely null, only all_null_action is applied. any_null_action is used only for partial missing values. A scalar ResultModel field that is null is treated as all-null.

The first error stops execution immediately.

column_policies shape:

• TableData / ResultModel: {column: policy}

• TableCollection: {table: {column: policy}}

Inherits from:

PipeModule

Methods:

__init__(mname: str = 'NullValueChecker', auto_run: bool = True, table: PortTypeHint.TableData | PortTypeHint.TableCollection | PortTypeHint.ResultModel | None = None, column_policies: ColumnPolicies | dict | None = None, empty_table_action: IssueAction = 'error', empty_table_message: str | None = None, treat_empty_string_as_null: bool = True, treat_empty_list_as_null: bool = True) → None

Initialize NullValueChecker.

Parameters

InputTablePortReference[TableData | TableCollection | ResultModel]

Input data to check.

OutputTablePortReference[TableData | TableCollection | ResultModel]

The same input object, unchanged.

execute() → PortTypeHint.TableData | PortTypeHint.TableCollection | PortTypeHint.ResultModel | None

Attributes:

InputTable: PortReference[PortTypeHint.TableData | PortTypeHint.TableCollection | PortTypeHint.ResultModel]

OutputTable: PortReference[PortTypeHint.TableData | PortTypeHint.TableCollection | PortTypeHint.ResultModel]