Skip to main content

Getting Started with Connectors

A connector knows how to read/write records from/to a data source/destination (e.g. a database).

When thinking about connectors for Conduit, our goals were to:

  • provide a good development experience to connector developers,
  • ship Conduit with real built-in connectors (compiled into the Conduit binary),
  • to make it as easy as possible to write plugins in any programming language,
  • the Connector SDK to be decoupled from Conduit and be able to change without changing Conduit itself.

Conduit ships with a number of built-in connectors:

  • File connector provides a source/destination to read/write a local file (useful for quickly trying out Conduit without additional setup).
  • Kafka connector provides a source/destination for Apache Kafka.
  • Postgres connector provides a source/destination for PostgreSQL.
  • S3 connector provides a source/destination for AWS S3.
  • Generator connector provides a source which generates random data (useful for testing).
  • Log connector provides a destination which outputs records in the Conduit logs.

Besides these connectors there is a number of standalone connectors that can be added to Conduit as plugins (find the complete list here).

Have a look at how to install a connector next!