Getting Started with Connectors

A Conduit Connector knows how to read/write records from/to a data source/destination (e.g. a database).

When thinking about connectors for Conduit, our goals were to:

• provide a good development experience to connector developers,
• ship Conduit with real built-in connectors (compiled into the Conduit binary),
• to make it as easy as possible to write plugins in any programming language,
• the Connector SDK to be decoupled from Conduit and be able to change without changing Conduit itself.

Conduit ships with a number of built-in connectors:

• File connector provides a source/destination to read/write a local file (useful for quickly trying out Conduit without additional setup).
• Kafka connector provides a source/destination for Apache Kafka.
• Postgres connector provides a source/destination for PostgreSQL.
• S3 connector provides a source/destination for AWS S3.
• Generator connector provides a source which generates random data (useful for testing).
• Log connector provides a destination which outputs records in the Conduit logs.

Besides these connectors there is a number of standalone connectors that can be added to Conduit as plugins (find the complete list here).

Have a look at how to install a connector next!