Skip to main content

webhook.http

Trigger an HTTP request for every record.

Description​

A processor that sends an HTTP request to the specified URL, retries on error and saves the response body and, optionally, the response status.

Configuration parameters​

version: 2.2
pipelines:
  - id: example
    status: running
    connectors:
      # define source and destination ...
    processors:
      - id: example
        plugin: "webhook.http"
        settings:
          # Maximum number of retries for an individual record when backing off
          # following an error.
          # Type: float
          backoffRetry.count: "0"
          # The multiplying factor for each increment step.
          # Type: float
          backoffRetry.factor: "2"
          # The maximum waiting time before retrying.
          # Type: duration
          backoffRetry.max: "5s"
          # The minimum waiting time before retrying.
          # Type: duration
          backoffRetry.min: "100ms"
          # Specifies which field from the input record should be used as the
          # body in the HTTP request.
          # For more information about the format, see [Referencing
          # fields](https://conduit.io/docs/processors/referencing-fields).
          # Type: string
          request.body: ""
          # The value of the `Content-Type` header.
          # Type: string
          request.contentType: "application/json"
          # Method is the HTTP request method to be used.
          # Type: string
          request.method: "GET"
          # URL is a Go template expression for the URL used in the HTTP
          # request, using Go [templates](https://pkg.go.dev/text/template). The
          # value provided to the template is
          # [opencdc.Record](https://github.com/ConduitIO/conduit-commons/blob/59ecfbe5d5be2ac4cd9a674d274862d164123f36/opencdc/record.go#L30),
          # so the template has access to all its fields (e.g. .Position, .Key,
          # .Metadata, and so on). We also inject all template functions
          # provided by [sprig](https://masterminds.github.io/sprig/) to make it
          # easier to write templates.
          # Type: string
          request.url: ""
          # Specifies in which field should the response body be saved.
          # For more information about the format, see [Referencing
          # fields](https://conduit.io/docs/processors/referencing-fields).
          # Type: string
          response.body: ".Payload.After"
          # Specifies in which field should the response status be saved. If no
          # value is set, then the response status will NOT be saved.
          # For more information about the format, see [Referencing
          # fields](https://conduit.io/docs/processors/referencing-fields).
          # Type: string
          response.status: ""

Examples​

Send a request to an HTTP server​

This example shows how to use the HTTP processor to send a record's .Payload.After field to a dummy HTTP server that replies back with a greeting.

The record's .Payload.After is overwritten with the response. Additionally, the example shows how to store the value of the HTTP response's code in the metadata field http_status.

Configuration parameters​

version: 2.2
pipelines:
  - id: example
    status: running
    connectors:
      # define source and destination ...
    processors:
      - id: example
        plugin: "webhook.http"
        settings:
          backoffRetry.count: "0"
          backoffRetry.factor: "2"
          backoffRetry.max: "5s"
          backoffRetry.min: "100ms"
          request.body: ".Payload.After"
          request.contentType: "application/json"
          request.method: "GET"
          request.url: "http://127.0.0.1:54321"
          response.body: ".Payload.After"
          response.status: ".Metadata["http_status"]"

Record difference​

Before
After
1
{
1
{
2
  "position": "cG9zLTE=",
2
  "position": "cG9zLTE=",
3
  "operation": "update",
3
  "operation": "update",
4
-
  "metadata": null,
4
+
  "metadata": {
5
+
    "http_status": "200"
6
+
  },
5
  "key": null,
7
  "key": null,
6
  "payload": {
8
  "payload": {
7
    "before": null,
9
    "before": null,
8
-
    "after": "world"
10
+
    "after": "hello, world"
9
  }
11
  }
10
}
12
}




Send a request to an HTTP server with a dynamic URL​


This example shows how to use the HTTP processor to use a record's .Payload.After.name field in the URL path,
send it to a dummy HTTP server, and get a greeting with the name back.


The response will be written under the record's .Payload.After.response.


Configuration parameters​


version: 2.2
pipelines:
  - id: example
    status: running
    connectors:
      # define source and destination ...
    processors:
      - id: example
        plugin: "webhook.http"
        settings:
          backoffRetry.count: "0"
          backoffRetry.factor: "2"
          backoffRetry.max: "5s"
          backoffRetry.min: "100ms"
          request.contentType: "application/json"
          request.method: "GET"
          request.url: "http://127.0.0.1:54321/{{.Payload.After.name}}"
          response.body: ".Payload.After.response"


Record difference​


Before
After
1
{
1
{
2
  "position": "cG9zLTE=",
2
  "position": "cG9zLTE=",
3
  "operation": "create",
3
  "operation": "create",
4
  "metadata": null,
4
  "metadata": null,
5
  "key": null,
5
  "key": null,
6
  "payload": {
6
  "payload": {
7
    "before": null,
7
    "before": null,
8
    "after": {
8
    "after": {
9
-
      "name": "foo"
9
+
      "name": "foo",
10
+
      "response": "aGVsbG8sIGZvbyE="
10
    }
11
    }
11
  }
12
  }
12
}
13
}


scarf pixel conduit-site-docs-processors