Data Processing Nodes¶

Nodes for transforming, extracting, and reshaping data.

Data Shaper¶

Reshapes data between different structures using field mappings and optional JSON Schema validation. This is the primary node for converting data between formats when connecting nodes with different data shapes.

Category: Data Processing

Input Ports¶

Port	Type	Required	Description
data	Mixed	Yes	The source data to reshape

Configuration¶

Parameter	Type	Description
mapping	Object	Field mapping configuration — maps source paths to target paths
schema	Object	Optional JSON Schema for validating the output
validationMode	String	`strict` (fail on invalid) or `soft` (warn but continue)

Output Ports¶

Port	Type	Description
data	Mixed	The reshaped data
valid	Boolean	Whether the output passed schema validation
errors	Array	Validation errors (if any)
mappedFields	Number	Number of fields successfully mapped

Usage¶

Use Data Shaper when you need to transform data from one structure to another — for example, extracting specific fields from a complex object, renaming fields, or restructuring nested data.

Data Extractor¶

Extracts specific values from complex data structures using JSONPath (RFC 9535) or dot-notation property paths.

Category: Data Processing

Input Ports¶

Port	Type	Required	Description
data	Mixed	Yes	The data to extract from

Configuration¶

Parameter	Type	Required	Description
path	String	Yes	JSONPath expression (e.g., `$.users[0].name`) or dot notation (e.g., `users.0.name`)
default	Mixed	No	Default value if the path doesn't match
extract_all	Boolean	No	Extract all matches (true) or just the first (false)

Output Ports¶

Port	Type	Description
data	Mixed	The extracted value(s)
path	String	The extraction path used
success	Boolean	Whether the extraction found a match
match_count	Number	Number of matches found

Common JSONPath Expressions¶

Expression	Description
`$.name`	Top-level field
`$.users[0].email`	First user's email
`$.users[*].email`	All users' emails
`$.items[?(@.active)]`	Items where `active` is true
`$..name`	All `name` fields at any depth

Data Operations¶

Performs common array operations on data: filtering, sorting, grouping, and more.

Category: Data Processing

Input Ports¶

Port	Type	Required	Description
data	Array	Yes	The array to operate on

Configuration¶

Parameter	Type	Required	Description
operation	String	Yes	Operation: `filter`, `sort`, `group`, `map`, `reduce`, `unique`, `slice`, `merge`
key	String	Varies	The field/key to operate on
value	Mixed	Varies	The value for comparison operations
condition	Mixed	Varies	Condition for filter operations

Output Ports¶

Port	Type	Description
result	Array	The operation result
operation	String	The operation performed
input_count	Number	Input array size
output_count	Number	Output array size

Split Text¶

Splits text into chunks using various methods.

Category: Data Processing

Input Ports¶

Port	Type	Required	Description
text	String	Yes	The text to split

Configuration¶

Parameter	Type	Default	Description
method	String	`words`	Split method: `words`, `characters`, `sentences`, `paragraphs`
chunkSize	Number	(varies)	Number of units per chunk
separator	String	(auto)	Custom separator string

Output Ports¶

Port	Type	Description
chunks	Array	The resulting text chunks
method	String	The split method used
chunk_size	Number	The chunk size used
total_chunks	Number	Total number of chunks

Regex Extractor¶

Extracts data using regular expressions with security protections against ReDoS (catastrophic backtracking).

Category: Data Processing

Input Ports¶

Port	Type	Required	Description
text	String	Yes	The text to search

Configuration¶

Parameter	Type	Required	Description
pattern	String	Yes	The regular expression pattern
flags	String	No	Regex flags (e.g., `i` for case-insensitive)
matchAll	Boolean	No	Find all matches (true) or just the first (false)

Output Ports¶

Port	Type	Description
matches	Array	Matched values
total_matches	Number	Number of matches found

Data to JSON¶

Converts structured data (objects, arrays) to a JSON string.

Category: Data Processing

Input Ports¶

Port	Type	Required	Description
data	Mixed	Yes	The data to serialize

Output Ports¶

Port	Type	Description
json	String	The JSON string
success	Boolean	Whether serialization succeeded

JSON to Data¶

Parses a JSON string into structured data (objects, arrays).

Category: Data Processing

Input Ports¶

Port	Type	Required	Description
json	String	Yes	The JSON string to parse

Output Ports¶

Port	Type	Description
data	Mixed	The parsed data structure
success	Boolean	Whether parsing succeeded

Data to Dataframe¶

Converts data into dataframe format with multiple output orientations.

Category: Data Processing

Input Ports¶

Port	Type	Required	Description
data	Array	Yes	The input data

Configuration¶

Parameter	Type	Default	Description
format	String	`json`	Output format: `json`, `csv`, `parquet`
includeIndex	Boolean	false	Include row index
orient	String	`records`	Orientation: `records`, `index`, `columns`, `split`

Output Ports¶

Port	Type	Description
dataframe	Mixed	The dataframe output
format	String	The format used
rows_count	Number	Number of rows
columns_count	Number	Number of columns

Message to Data¶

Converts text messages to structured data by parsing JSON, CSV, XML, YAML, or key-value format.

Category: Data Processing

Input Ports¶

Port	Type	Required	Description
message	String	Yes	The message text to parse

Configuration¶

Parameter	Type	Default	Description
format	String	`json`	Expected format: `json`, `csv`, `xml`, `yaml`, `key_value`
extractFields	Array	(none)	Specific fields to extract from the parsed data

Output Ports¶

Port	Type	Description
data	Object	The parsed structured data
format	String	The format used
original_message	String	The original message text