The rejected rows are the dataset that tells you what is wrong upstream
When pipelines clean data, they discard anything they cannot process. The clean table looks fine. It does not explain why 11% of the rows failed.
Those failures point to a problem in the upstream system. When the rejection rate on a particular field goes up, a form has changed, an integration has broken, or a new supplier is sending data in a different format. The clean table does not show that.
Duplicate rows also need care. Two rows that look the same might be two real events. A deduplication step that automatically combines them removes a number that nobody can reconstruct.
The judgment call is the point where a rule gives way to interpretation. A rule can parse a badly formatted date mechanically. A company name in three different spellings requires a decision about whether the variants refer to the same company.
How the ActionFlow runs on the canvas
Raw rows enter through a Document Upload node, a Google Sheets node, or an HTTP Request node that retrieves them from the source system.
The Validate node acts first. It does the mechanical work: checking types, verifying required fields, and validating format. If a rule can decide something, this node handles it. Nothing is passed to a model when a rule could have handled it.
A Filter node separates valid rows from invalid ones. It does not drop the invalid rows. It places them in their own table, with the failed field listed on each row.
A Uniq node removes exact duplicates. A Sort node then arranges the rest so that near-duplicates sit next to each other.
A Generate Object node handles only the cases that survived and still look ambiguous: variant spellings, inconsistent categories, and values that are possible but unusual. It does not rewrite the rows. It proposes a normalization and a confidence level for each row.
A second Filter node uses the confidence threshold. High-confidence normalizations are added to the clean table through a PostgreSQL, MySQL, or Google Sheets node. The others go to a review table.
A Create Excel node produces a file that someone can work through offline as part of the review set.
Nodes this ActionFlow uses
- Document Upload (Data): takes the raw file. Google Sheets and HTTP Request cover other sources.
- Validate (Data): does everything a rule can decide, before any model is involved.
- Filter (Data): splits valid from invalid and keeps the invalid set with its failure reason.
- Uniq (Control): removes exact duplicates without merging genuine repeat events.
- Generate Object (AI Core): proposes normalizations with a confidence per row, rewriting nothing.
- PostgreSQL (Data): receives the clean table. MySQL and Google Sheets fit the same slot.
- Create Excel (Data): produces a file for the review set when someone works offline.
Frequently asked questions
Start building AI workflows
Create a free account, open a template or a blank canvas, and run your first ActionFlow.
Newsletter
Get product updates
New nodes, agents, and product notes. We send mail only when we have something worth opening.