modules.dataClean

Null-value checking for TableData, TableCollection, and ResultModel.

Classes

class modules.dataClean.NullValueChecker

Check specified columns for null values and react per column policy.

This module does not modify data. The input object is passed through unchanged. Null values are None, NaN, NaT, and pd.NA. Empty strings "" are also treated as null when treat_empty_string_as_null is True. Empty lists [] and empty tuples () are treated as null when treat_empty_list_as_null is True.

When a column or list field is entirely null, only all_null_action is applied. any_null_action is used only for partial missing values. A scalar ResultModel field that is null is treated as all-null.

When the whole input is None, none_input_action applies only after InputTable has been written. The automatic execute() at the end of initialization sees an unwritten port and does not emit. ignore, warning, and gdi_warning still pass None through. error raises ValueError.

The first error stops execution immediately, except when table_requirement is set: issues from alternative table groups are collected and only raised if every group fails.

column_policies shape:

• TableData / ResultModel: {column: policy}

• TableCollection: {table: {column: policy}}

Messages name a table or column by its own title when that object exists. NullCheckPolicy.display_column_title and display_table_titles are labels used only when the column, field, or table cannot be found. Without a user label, the message uses the policy key.

Inherits from:

PipeModule

Methods:

__init__(mname: str = 'NullValueChecker', auto_run: bool = True, table: PortTypeHint.TableData | PortTypeHint.TableCollection | PortTypeHint.ResultModel | None = None, column_policies: ColumnPolicies | dict | None = None, display_table_titles: dict[str, str] | None = None, table_requirement: TableRequirement | None = None, empty_table_action: IssueAction = 'error', empty_table_message: str | None = None, none_input_action: IssueAction = 'ignore', none_input_message: str | None = None, treat_empty_string_as_null: bool = True, treat_empty_list_as_null: bool = True) → None

Initialize NullValueChecker.

Raises

display_table_titles

dict[str, str] | None, optional Titles for TableCollection lookup keys, used only when that table is missing. Keys match column_policies or table_requirement. A table that exists keeps its own title.

A missing title falls back to the lookup key. Example

NullValueChecker( column_policies={

“bore_table”

{

“bore_number”

{“all_null_action”: “ignore”}, } },

display_table_titles={“bore_table”

“钻孔一览表”}, ) A missing bore_table raises

``KeyError

“指定的表 ‘钻孔一览表’ 不存在”. Without ``display_table_titles, that message uses bore_table. If the table exists, messages use its own title, such as 钻孔表.

table_requirement

list[list[str]] | None, optional TableCollection only. OR-of-AND groups, e.g. [["A"], ["B", "C"]] means A or (B and C). Keys may be table names or table titles. A group passes only if every table exists and no error / warning / gdi_warning is triggered. The first passing group wins and later groups are skipped. If every group fails, issues from groups whose tables all existed are raised together. KeyError is raised only when every group failed because a table was missing. Tables in column_policies but not in table_requirement are always required (missing → KeyError). Must be a 2D list; a flat list like ["A", "B"] is rejected.

empty_table_action

{“error”, “warning”, “gdi_warning”, “ignore”}, default “error” Action when a checked table has zero rows. Column null checks are skipped for that table after this action (except error, which stops immediately).

empty_table_message

str, optional

Custom message for the empty-table case. Placeholder

{table}.

none_input_action

{“ignore”, “warning”, “gdi_warning”, “error”}, default “ignore” Action when InputTable was written and its value is None. The unwritten None present at construction does not trigger this, so initialization does not warn. ignore, warning, and gdi_warning still pass None through. error raises ValueError.

none_input_message

str, optional Custom message for a written None input, used as written. When omitted, the default message names the upstream module and port.

treat_empty_string_as_null

bool, default True Treat "" as a null value. Whitespace-only strings are not treated as null.

treat_empty_list_as_null

bool, default True Treat [] and () as null values. A ResultModel field whose value is [] or () is treated as all-null. Ports

InputTable

PortReference[TableData | TableCollection | ResultModel] Input data to check.

OutputTable

PortReference[TableData | TableCollection | ResultModel] The same input object, unchanged.

execute() → PortTypeHint.TableData | PortTypeHint.TableCollection | PortTypeHint.ResultModel | None

Attributes:

InputTable: PortReference[PortTypeHint.TableData | PortTypeHint.TableCollection | PortTypeHint.ResultModel]

OutputTable: PortReference[PortTypeHint.TableData | PortTypeHint.TableCollection | PortTypeHint.ResultModel]