Getting started with Connectors
A connector knows how to read/write records from/to a data source/destination (e.g. a database).
When thinking about connectors for Conduit, our goals were to:
- provide a good development experience to connector developers,
- ship Conduit with real built-in connectors (compiled into the Conduit binary),
- to make it as easy as possible to write plugins in any programming language,
- the Connector SDK to be decoupled from Conduit and be able to change without changing Conduit itself.
Conduit ships with a number of built-in connectors:
- File connector provides a source/destination to read/write a local file (useful for quickly trying out Conduit without additional setup).
- Kafka connector provides a source/destination for Apache Kafka.
- Postgres connector provides a source/destination for PostgreSQL.
- S3 connector provides a source/destination for AWS S3.
- Generator connector provides a source which generates random data (useful for testing).
- Log connector provides a destination which outputs records in the Conduit logs.
Besides these connectors there is a number of standalone connectors that can be added to Conduit as plugins (find the complete list here).
Have a look at how to install a connector next!