Installing and running
- Homebrew
- Using a pre-built binary
- Docker
- Building from source
The recommended way of running Conduit on a local machine is using the latest version of Conduit available on Homebrew which you can install by running the following command:
brew install conduit
Run
Once installed, you can start Conduit by running:
conduit
Below we'll walk through getting Conduit installed and running on your system.
Download
First, download the latest Conduit release for your platform.
For an ARM based back Mac for example we'd grab the conduit_0.11.1_Darwin_arm64.tar.gz
archive.
unzip the archive
tar xzvf conduit_0.11.1_Darwin_arm64.tar.gz
Run
In the directory in which you expanded the archive. Execute the Conduit binary:
./conduit
We distribute Docker images to ease the use of Conduit in a containerized environment. To see details about how our images are built please feel free to take a look at the Dockerfile
here.
Conduit's Docker images are hosted on GitHub.
Run
After you install Docker, you can pull the Conduit image by running the following command:
docker pull conduit.docker.scarf.sh/conduitio/conduit
Then, you can run Conduit by running:
docker run -it -p 8080:8080 conduit.docker.scarf.sh/conduitio/conduit
The command aboves runs Conduit with no pipelines. To run Conduit with a custom pipeline file you can mount it as a volume into the container:
docker run -it -p 8080:8080 \
-v /path/to/pipeline.yaml:/app/pipelines/pipeline.yaml \
conduit.docker.scarf.sh/conduitio/conduit
If you're using Docker, check out Watchtower. After we push an update, Watchtower will keep your Conduit container updated by gracefully shutting down your existing container and restarting it with the same options used when initially deployed.
Before you can build Conduit from source, you need to have Go 1.22 or higher installed. The Go website has comprehensive instructions on how to get the language installed on your machine. Once installed, you can follow the rest of the instructions.
- Start by downloading the source code from the latest stable release on the Conduit Releases Page. Alternatively, you can run this command to automatically download the latest stable source to your current directory:
$ TAG=v0.11.1; curl -o conduit.tar.gz -L https://github.com/ConduitIO/conduit/archive/refs/tags/$TAG.tar.gz
A file called conduit.tgz
will be in your current directory. The next step is to expand the source:
$ tar zxvf conduit.tgz
Then change directories to the appropriate folder. Keep in mind that the folder name might be different between releases since it's tied to the latest git sha for the commit.
$ cd conduit-0.11.1
Now build the project:
$ make
If you want to build Conduit without the UI, instead of running make
, you can run the following command:
$ make build-server
You will have a new binary built for your architecture and machine. All that's left is to run it!
./conduit
You should now be able to interact with the Conduit UI and HTTP API on port 8080:
....
.::::::::::.
.:::::‘‘‘‘:::::.
.:::: ::::.
.:::::::: ::::::::.
`:::::::: ::::::::‘
`:::: ::::‘
`:::::....:::::‘
`::::::::::‘ Conduit v0.11.1 linux/amd64
‘‘‘‘
2024-02-20T21:37:45+00:00 INF All 0 tables opened in 0s component=badger.DB
2024-02-20T21:37:45+00:00 INF Discard stats nextEmptySlot: 0 component=badger.DB
2024-02-20T21:37:45+00:00 INF Set nextTxnTs to 0 component=badger.DB
2024-02-20T21:37:45+00:00 INF builtin plugins initialized component=builtin.Registry count=6
2024-02-20T21:37:45+00:00 WRN could not read plugin directory error="open /app/connectors: no such file or directory" component=standalone.Registry
2024-02-20T21:37:45+00:00 INF standalone plugins initialized component=standalone.Registry count=0 plugin_path=/app/connectors
2024-02-20T21:37:45+00:00 INF processors initialized component=processor.Service count=0
2024-02-20T21:37:45+00:00 INF connectors initialized component=connector.Service count=0
2024-02-20T21:37:45+00:00 INF pipelines initialized component=pipeline.Service count=0
2024-02-20T21:37:45+00:00 INF pipeline configs provisioned component=provisioning.Service created=[] deleted=[] pipelines_path=./pipelines
2024-02-20T21:37:45+00:00 INF grpc server started address=[::]:8084
2024-02-20T21:37:45+00:00 INF http server started address=[::]:8080
2024-02-20T21:37:45+00:00 INF
2024-02-20T21:37:45+00:00 INF click here to navigate to Conduit UI: http://localhost:8080/ui
2024-02-20T21:37:45+00:00 INF click here to navigate to explore the HTTP API: http://localhost:8080/openapi
2024-02-20T21:37:45+00:00 INF
Next Steps
Now that you have Conduit installed you can learn how to build a pipeline. You can also explore some other topics, such as: