Skip to main content

Conduit v0.14.0 release

· One min read

We're happy to announce the release of Conduit v0.14.0, introducing the split and clone processors, which enables one source record to trigger a write of multiple records in the destination.

tip

The new processors return multiple records, which is currently only supported in the new v2 engine. If you want to try them out, you need to start Conduit using conduit run --preview.pipeline-arch-v2.

Key Highlights​

  • New option --connectors.max-receive-record-size in conduit run to configure the record size when using standalone connectors. More information.
  • Processor split enables you to split an array in a record into separate records. This can be useful in AI pipelines where the input record contains a document and the output is multiple embeddings inserted into a database.
  • Processor clone enables you to clone any record N times. This is useful when one input record needs to be split into multiple collections (tables) on the destination.
  • Improvements for pipelines init command, which now creates a runnable demo pipeline when no flags are supplied.

scarf pixel conduit-site-changelog