Worker Bridge

Worker-Bridge, an integral component of the Golem OSS worker-service, empowers you to define and upload API definitions. These definitions allow you to expose endpoints to users and specify how incoming requests should be managed by your Golem worker workflow/application.

Let's dig a bit deeper using an example.

Say you already have a worker running in your Golem OSS (docker environment) or Golem Cloud. Assuming you have already got the worker running in Golem with shopping-cart example, let's say we have to expose an endpoint to get the cart contents for a user. In order to do this, you would typically write an API definition as given below and simply upload to Golem.

At this point, worker-bridge is acting as an API-Gateway, but the major difference here is, it is geared to work seamlessly with Golem. In other words, we don't need to specifically write a custom backend service as worker-bridge gives it for free.

Before we begin, let's assume you uploaded a shopping-cart component and it list the following functions. We are using golem-cli in all these examples now on:

 golem-cli component list
[
  {
    "componentId": "499e5682-e29f-419c-ab68-e3a8f42c022c",
    "componentVersion": 0,
    "componentName": "test-component",
    "componentSize": 116419,
    "exports": [
      "golem:it/api/initialize-cart(user-id: string)",
      "golem:it/api/add-item(item: record { product-id: string, name: string, price: float32, quantity: u32 })",
      "golem:it/api/remove-item(product-id: string)",
      "golem:it/api/update-item-quantity(product-id: string, quantity: u32)",
      "golem:it/api/checkout() -> variant { error(string), success(record { order-id: string }) }",
      "golem:it/api/get-cart-contents() -> list<record { product-id: string, name: string, price: float32, quantity: u32 }>",
      "golem:it/api/not-durable()"
    ]
  }
]

Here is a simple API definition that allows you to expose get-cart-contents in the worker as an http endpoint.

{
  "id": "my-shopping-cart-v1",
  "draft": true,
  "version": "0.0.2",
  "routes": [
    {
      "method": "Get",
      "path": "/{user-id}/get-cart-contents",
      "binding": {
        "type": "wit-worker",
        "componentId": "499e5682-e29f-419c-ab68-e3a8f42c022c",
        "workerName": "my-worker-${request.path.user-id}",
        "response": "${let result = golem:it/api.{get-cart-contents}(); {status: 200, body: result}}"
      }
    }
  ]
}

Before we upload and deploy this API definition, let's break this down.

id: This field represents the unique identifier for the API definition. In this case, it is set to "shopping-cart-v1".

version: This field indicates the version of the API definition. Here, it is set to "0.0.3".

routes: This field contains an array of route objects, each representing a specific endpoint definition.

method: Indicates the HTTP method associated with the route. In this example, it is set to "Get", indicating that this route handles GET requests.

path: Specifies the URL path pattern for the route. It may include path parameters enclosed in curly braces. Here, the path is /{user-id}/get-cart-contents, indicating that this route handles requests to retrieve the contents of a shopping cart for a specific user.

binding: This object contains information about how the request should be handled by the Golem worker.

Golem Worker Binding

Let's break down the binding object.

type: Specifies the type of binding. Here, it is set to "wit-worker", indicating that the binding involves a Golem worker. Currently, the only supported type is "wit-worker".

componentId: Provides the component ID associated with the worker binding. As you can see the the golem-worker

workerName: Specifies the Name of the Golem worker responsible for handling the request. Here it is my-worker-${request.path.user-id}. The value that is wrapped in code block (starting with ${ and ending with }) will be a Rib expression. Here request.path.user-id is a valid Rib expression where it selects the field path from the request block, and further selects user-id from it.

Note that anything wrapped in this block is an Rib Expression which gets evaluated at runtime and in this case, it gets evaluated to actual user-id which is then concatenated with the text "my-worker-". If you pass complex Rib expressions to be concatenated with a worker-id (for example: It is evaluated to a WASM record) then you will get a response back saying invalid worker-name.

response: The response field here is a Rib expression (therefore wrapped in a code block) that allows you to call any worker function and manipulates its output. In this example, the following is the Rib expression:

let result = golem:it/api.{get-cart-contents}();
{status: 200, body: result}

The first line is about calling the worker-function and assigning its reuslt to the variable result. The last line is WASM record where you are mapping to the response. Here body is result itself. That means, we are not manipulating the result returned by the function, and simply forward it as response body. Status is 200. There are various possibilities here to manipulate the data using Rib language, such as selecting only a particular field from the result of the function invocation. Say for example, the result is a WASM result which can be ok or err. In this case you can have a complex Rib expression as below

let result = golem:it/api.{get-cart-contents}();

let response_status = match result {
  ok(_) => 200,
  err(_) => 400
};

let response_body = match result {
  ok(result) => result,
  err(msg) => msg
}

{status: response_status, body: response_body}

The result of the Rib expression is whatever the result of the last line/expression in the above Rib program, similar to many programming languages.

Upload API definition

You can either use Golem-CLI or REST endpoints or Golem-UI (available only in golem-cloud) to upload the API definition to Golem.

Here is how you upload API definition to the worker-bridge. Save the above definition in a file. Let's say, api-definition.json

Terminal

golem-cli api-definition add api-definition.json

This returns:

 [
  {
    "id": "my-shopping-cart-v1",
    "version": "0.0.2",
    "routes": [
      {
        "method": "Get",
        "path": "/{user-id}/get-cart-contents",
        "binding": {
          "componentId": "499e5682-e29f-419c-ab68-e3a8f42c022c",
          "workerName": "my-worker-${request.path.user-id}",
          "idempotencyKey": null,
          "response": "${let result = {golem:it/api}.get-cart-contents();\n{status: 200, body: result}}"
        }
      }
    ],
    "draft": true
  }
 ]

Here the draft is true is since it is still not deployed and have the endpoint of get-cart-contents exposed. Once it is deployed, the value of the field draft becomes false, and you cannot further update this definition unless you change the version. This is because, you really don't want to update a definition from outside, while it is already exposed an endpoint to the users.

As mentioned before, you can upload an API definition directly using REST APIs.

Terminal

curl -X POST http://localhost:9881/v1/api/definitions -H "Content-Type: application/json" -d @api-definition.json

Here localhost:9881 is where worker-service (that consist of worker-bridge functionality) is running, ready to accept these API management requests. The docker-compose example (explained in quick-start) currently exposes the port 9881 for worker-service.

Deploy the API Definition

Using golem-cli, deploying the API definition is straight forward.

Terminal

golem-cli api-deployment deploy --definition my-shopping-cart-v1/0.0.2 --host localhost:9006

The above command returns the following:

{
  "apiDefinitions": [
    {
      "id": "my-shopping-cart-v1",
      "version": "0.0.2"
    }
  ],
  "site": {
    "host": "localhost:9006",
    "subdomain": null
  }
}

Here we deploy the definition and now the endpoints should be available at host localhost:9006. If you were following docker examples to spin up Golem, then the port has to be 9006. Refer to .env in docker-examples folder in Golem OSS.

You can test the deployment now, by trying to call the endpoint defined in the API definition.

curl -H "Accept: application/json" -X GET http://localhost:9006/afsal/get-cart-contents
[]

Alternatively you can use REST APIs to deploy the API definition.

Let's create a deployment.json as given below. Based on the example above, the apiDefinitionId is shopping-cart-v1 and version is 0.0.2.

If you were having a domain registered such as my-site.com, you would replace localhost:9006 with my-site.com, that in turn redirects to localhost:9006, or wherever the worker-service is running (perhaps AWS).

{
  "apiDefinitionId": "shopping-cart-v1",
  "version": "0.0.3",
  "site": "localhost:9006"
}

Terminal

 
curl -X POST http://localhost:9881/v1/api/deployments/deploy -H "Content-Type: application/json"  -d @deployment.json

Start using the API

As you have seen before, after the deployment, we call the endpoint.

Terminal

 
curl -X GET http://localhost:9006/123/get-cart-contents

Now worker-bridge identifies the user to be 123 and evaluates the worker-name expr to be worker-123. It then forwards the request to the worker with id worker-123 and invokes the function golem:it/api.{get-cart-contents} with empty parameters, and simply get the worker-response (which is a WASM value) and converts it to Json and sends it back to the client.

Response Mapping

Response Mapping is under the response field under each binding. This is written using Rib to call the worker function and manipulate it's results and return a value that's understandable for each protocol we use.

Here is a more complex example of an API definition that gives you more idea on what can be achieved with Rib expression under response field.

{
  "id": "my-shopping-cart-v2",
  "draft": true,
  "version": "2.0.11",
  "routes": [
    {
      "method": "Get",
      "path": "/{user-id}/get-cart-contents",
      "binding": {
        "type": "wit-worker",
        "componentId": "5ac3aea2-8a80-40f6-ac20-f24307fd799b",
        "workerName": "my-worker-${request.path.user-id}",
        "response" : "${let result = golem:it/api.{get-cart-contents}(); {status: 200, body: result}}"
      }
    },
    {
      "method": "Post",
      "path": "/{user-id}/initialize-cart",
      "binding": {
        "type": "wit-worker",
        "componentId": "5ac3aea2-8a80-40f6-ac20-f24307fd799b",
        "workerName": "my-worker-${request.path.user-id}",
        "response" : "${golem:it/api.{initialize-cart}(request.path.user-id); {status: 200, body: []}}"
      }
    },
     {
      "method": "Post",
      "path": "/{user-id}/add-item",
      "binding": {
        "type": "wit-worker",
        "componentId": "5ac3aea2-8a80-40f6-ac20-f24307fd799b",
        "workerName": "my-worker-${request.path.user-id}",
        "response" : "${golem:it/api.{add-item}({product-id: request.body.product-id, name: request.body.name, price: request.body.price, quantity: request.body.quantity}); {status: 200, body: []}}"
      }
     },
    {
      "method": "Post",
      "path": "/{user-id}/checkout",
      "binding": {
        "type": "wit-worker",
        "componentId": "5ac3aea2-8a80-40f6-ac20-f24307fd799b",
        "workerName": "my-worker-${request.path.user-id}",
        "response" : "${let result = golem:it/api.{checkout}(); {status: 200, body: match result { success(resp) => resp.order-id, error(msg) => msg }}}"
      }
    }
  ]
}

Try and deploy this version. Before we try and use these endpoints, let's point out a few details.

Here we used multiple functions in the shopping-cart components and exposed each one of them as an http endpoint. Most of them are self explanatory, but worth pointing out a few details.

The examples show how you can pass parameters to the function. This is more or less same as many programming languages - a comma separated list of arguments. The most interesting example out of these endpoints is the checkout endpoint.

Here the response mapping in Rib is this:

let result = golem:it/api.{checkout}();

{status: 200, body: match result { success(resp) => resp.order-id, error(msg) => msg }}

This can also be written as the following for more readability:

let result = golem:it/api.{checkout}();
let status = 200;
let body =  match result { success(resp) => resp.order-id, error(msg) => msg };
{status: status, body: body}

Here we manipulate the result from worker function using a match expression. More details on match expression is explained under Rib documentation. It matches on success and error variants because the result type of golem:it/api.{checkout} is a variant which can be either success or error.

Looking up data from the request body and path

In the above example, we have an endpoint to add products into the cart.

     {
      "method": "Post",
      "path": "/{user-id}/add-item",
      "binding": {
        "type": "wit-worker",
        "componentId": "5ac3aea2-8a80-40f6-ac20-f24307fd799b",
        "workerName": "my-worker-${request.path.user-id}",
        "response" : "${golem:it/api.{add-item}({product-id: request.body.product-id, name: request.body.name, price: request.body.price, quantity: request.body.quantity}); {status: 200, body: []}}"
      }
     }

This implies, the http request should be method POST with request body having the following fields: product-id, name, price and quantity. Similarly, yoiu can select field from the request path in Rib expression. We already used this to fetch user-id to form the worker name.

Test the API endpoints

Once you upload and deploy the API definition above, you can try the following:

With the following request body saved in a file request_body.json

{
  "product-id" : "foo",
  "name" : "iphone",
  "price" : 2500,
  "quantity": 1
}

curl -H "Accept: application/json" -X POST http://localhost:9006/afsal/initialize-cart
#[]


curl -H "Accept: application/json" -X POST http://localhost:9006/afsal/add-item -d @request_body.json

[]%

curl -H "Accept: application/json" -X GET http://localhost:9006/afsal/get-cart-contents
[{"name":"foo","price": 2500,"product-id":"foo","quantity":1}]%

curl -H "Accept: application/json" -X POST http://localhost:9006/afsal/checkout
#238738674%

Note that there is currently no way to express unit in Rib as Rib is mostly WASM-WAVE syntax. This is why for certain endpoints we simply returned []. You can also prefer to return an empty string "".

Support to import OpenAPI spec

Worker-bridge allows you to import API definition in OpenAPI spec format. This is because, users may have already written an OpenAPI spec for their endpoints for various purposes. By adding extra details of worker-bridge into the same spec, we can use it as an API definition. Internally it gets converted to the worker-bridge's native format of API definition, discussed in the beginning of this documentation.

The main advantage of this feature is the re-usability of the same endpoint definitions across your system. For example, you can use the same file now to register with worker-bridge and register with another external API gateway. More on this below. However once deployed, it returns native format then onwards, and updates needs to be done in the native format.

{
  "openapi": "3.0.0",
  "x-golem-api-definition-version": "0.0.3",
  "x-golem-api-definition-id": "shopping-cart-v1",
  "info": {
    "title": "Sample API",
    "version": "1.0.2"
  },
  "paths": {
    "/{user-id}/get-cart-contents": {
      "x-golem-worker-bridge": {
        "worker-name": "worker-${request.path.user-id}",
        "component-id": "2696abdc-df3a-4771-8215-d6af7aa4c408",
        "response": "${ { headers: { ContentType: \"json\", userid: \"foo\"}, body: golem:it/api.{get-cart-contents}(), status: 200 } }"
      },
      "get": {
        "summary": "Get Cart Contents",
        "description": "Get the contents of a user's cart",
        "parameters": [
          {
            "name": "user-id",
            "in": "path",
            "required": true,
            "schema": {
              "type": "string"
            }
          }
        ],
        "responses": {
          "200": {
            "description": "OK",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/CartItem"
                }
              }
            }
          },
          "404": {
            "description": "Contents not found"
          }
        }
      }
    }
  },
  "components": {
    "schemas": {
      "CartItem": {
        "type": "object",
        "properties": {
          "id": {
            "type": "string"
          },
          "name": {
            "type": "string"
          },
          "price": {
            "type": "number"
          }
        }
      }
    }
  }
}

To import OpenAPI spec using Golem CLI,

Terminal

 
golem-cli api-definition import @open_api_spec.json

Alternatively, you can use REST API endpoint

Terminal

curl -X POST http://localhost:9881/v1/api/definitions/oas -H "Content-Type: application/json"  -d @openapi_spec.json

Integrating Golem with existing API Gateways

Note that, while worker-bridge is a powerful tool for defining and managing API endpoints, it is not a full-fledged API Gateway. However, it can be used conjunction with existing API Gateways, allowing you to leverage the capabilities of both systems. For example, you can use worker-bridge to define and manage the worker bindings for your API endpoints, while using an API Gateway to handle other aspects of API management, such as authentication, rate limiting, and monitoring. In this scenario, the API Gateway would route incoming requests to worker-bridge based on the defined endpoints, allowing worker-bridge to handle the request processing and response generation.

Let's say you have registered the API definition with worker-bridge, and deployed with a site. Now let's integrate with Tyk API gateway.

Note that Tyk allows user to upload OpenAPI spec similar to worker-bridge. You can upload the same OpenAPI spec with worker-bridge info to Tyk, with 1 more extra information which is servers block with the value of the URL of worker-bridge, that tells the API gateway to route the request to worker-bridge. Obviously, it depends on how you installed Tyk. If you installed Tyk in the same network as worker-bridge, you can use the localhost as servers. If you are using a separate docker network with Tyk, you will need to give the machine IP address to reach the worker-bridge URL.

{
  "openapi": "3.0.0",
  "x-golem-api-definition-version": "0.0.3",
  "x-golem-api-definition-id": "shopping-cart-v1",
  "info": {
    "title": "Sample API",
    "version": "1.0.2"
  },
  "servers": [
    {
      "url": "http://ip-address-of-your-local-machine:9881"
    }
  ],
  "paths": {
    "/{user-id}/get-cart-contents": {
      "x-golem-worker-bridge": {
        "worker-name": "worker-${request.path.user-id}",
        "component-id": "2696abdc-df3a-4771-8215-d6af7aa4c408",
        "response": "${ { headers: { ContentType: \"json\", userid: \"foo\"}, body: golem:it/api.{get-cart-contents}(), status: 200 } }"
      },
      "get": {
        "summary": "Get Cart Contents",
        "description": "Get the contents of a user's cart",
        "parameters": [
          {
            "name": "user-id",
            "in": "path",
            "required": true,
            "schema": {
              "type": "string"
            }
          }
        ],
        "responses": {
          "200": {
            "description": "OK",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/CartItem"
                }
              }
            }
          },
          "404": {
            "description": "Contents not found"
          }
        }
      }
    }
  },
  "components": {
    "schemas": {
      "CartItem": {
        "type": "object",
        "properties": {
          "id": {
            "type": "string"
          },
          "name": {
            "type": "string"
          },
          "price": {
            "type": "number"
          }
        }
      }
    }
  }
}

The difference here is we added the server's block. You can include this servers block from the beginning so that it's exactly the same OpenAPI spec which you used to upload to worker-bridge as well as Tyk.

Install Tyk

Terminal

git clone https://github.com/TykTechnologies/tyk-gateway-docker
cd tyk-gateway-docker
docker-compose up

Registration with Tyk

Let's say we saved the above json as open_api.json

Terminal

curl -X POST http://localhost:8080/tyk/apis/oas/import --header 'x-tyk-authorization: foo' --header 'Content-Type: text/plain' -d @open_api.json

Reload the Tyk API Gateway, otherwise the API is not deployed with Tyk yet, so this is an important step. Note that, if you are encountering issues following these steps, please refer to Tyk documentations.

curl -H "x-tyk-authorization: foo" -s http://localhost:8080/tyk/reload/group

Note that Tyk is now running at 8080, and now requests has to go into 8080 and not worker-bridge

Terminal

curl -X GET http://localhost:9006/adam/get-cart-contents

How does worker service know which API definition to pick for a given endpoint?

When a request comes in, worker-service looks at the host in the request and matches it with the site in the deployment. If there is a deployment corresponding to the site, it picks the API definition ID and version from the deployment and gets the API definition, to further process the request

Updating Components