Configure message brokers
Akka Serverless integrates with Google Cloud Pub/Sub and Confluent Cloud to integrate with other services and to allow asynchronous messaging within Akka Serverless services.
Confluent Cloud
Take the following steps to configure access to your Confluent Cloud Kafka broker for your Akka Serverless project:app-name:
-
Open Confluent Cloud.
-
Go to your cluster. Create a new cluster if you don’t have one already.
-
Go to
data integration
,clients
. Create a new client with theNew client
button. -
Choose
Java
orScala
. Create a key with theCreate Kafka cluster API key
button. Copy the configuration shown in the user interface into a file, and save it to <kafka-broker-config-file>. -
Use akkasls to configure the broker for your project:
$ akkasls projects set broker --broker-service kafka --broker-config-file <kafka-broker-config-file>
Create a topic
To create a topic, you can either use the Confluent Cloud user interface, or the Confluent Cloud CLI.
- Browser
-
-
Open Confluent Cloud.
-
Go to your cluster
-
Go to the Topics page
-
Use the Add Topic button
-
Fill in the topic name, select the number of partitions, and use the Create with defaults button
You can now use the topic to connect with Akka Serverless.
-
- Confluent Cloud CLI
-
ccloud kafka topic create TOPIC_ID
You can now use the topic to connect with Akka Serverless.
Use a Kafka broker locally
The docker-compose example file below shows how to set the akkaserverless.proxy.eventing.support
to kafka
.
A volume is mounted to the current directory .
, which is expected to contain a kafka.properties
file.
The BROKER_CONFIG_FILE
environment variable points to the kafka.properties
file in the mounted volume 'conf', which
ensures that the service can connect to Kafka.
version: "3"
services:
akka-serverless-proxy:
image: gcr.io/akkaserverless-public/akkaserverless-proxy:0.8.11
command: -Dconfig.resource=dev-mode.conf -Dakkaserverless.proxy.eventing.support=kafka
ports:
- "9000:9000"
extra_hosts:
- "host.docker.internal:host-gateway"
environment:
USER_FUNCTION_HOST: ${USER_FUNCTION_HOST:-host.docker.internal}
USER_FUNCTION_PORT: ${USER_FUNCTION_PORT:-8080}
BROKER_CONFIG_FILE: /conf/kafka.properties
volumes:
- .:/conf
You can check out a complete example in Java Customer Registry (with Kafka).
Google Cloud Pub/Sub
To configure access to your Google Cloud Pub/Sub broker for your Akka Serverless project, you need to create a Google service account with access to your Google Cloud Pub/Sub broker and provide it to Akka Serverless.
Details on doing this can be found in the Google documentation. We provide simplified steps below.
The service account should allow for the roles/pubsub.editor
role.
Setting up the service account
To set up a service account and generate the key, follow these steps:
-
Navigate to https://console.cloud.google.com/
.
-
From the blue bar, click the dropdown menu next to Google Cloud Platform.
-
Click New Project to create a project and save the
<gcp-project-id>
, which you will need later. -
Enter the following
gcloud
command to set up thegcloud
environment:gcloud auth login gcloud projects list gcloud config set project <gcp-project-id>
-
Enter the following command to create the service account. The example uses the name
akka-serverless-broker
, but you can use any name.gcloud iam service-accounts create akka-serverless-broker
-
Enter the following commands to grant the GCP Pub/Sub editor role to the service account. Substitute your project ID for
<gcp-project-id>
.gcloud projects add-iam-policy-binding <gcp-project-id> \ --member "serviceAccount:akka-serverless-broker@<gcp-project-id>.iam.gserviceaccount.com" \ --role "roles/pubsub.editor"
-
Generate a key file for your service account:
gcloud iam service-accounts keys create keyfile.json \ --iam-account akka-serverless-broker@<gcp-project-id>.iam.gserviceaccount.com
-
Use the cat command to invoke the keyfile.json content:
cat keyfile.json
-
Copy the content of the file to your clipboard.
Now you have a service account key file to configure Akka Serverless to use your Google Cloud Pub/Sub broker. Use the following steps to add the key file from the AkkaServerless console:
- Browser
-
-
Open the project in the AkkaServerless Console.
-
Select Integrations from the left-hand navigation menu.
-
Click + for the Google Cloud Pub/Sub integration option.
-
Copy the contents of
keyfile.json
into the editor and, click Apply.
The project is now configured to use Google Pub/Sub as the message broker.
-
- CLI
-
akkasls projects set broker --broker-service google-pubsub --gcp-key-file keyfile.json
The project is now configured to use Google Pub/Sub as the message broker.
Create a topic
To create a topic, you can either use the Google Cloud Console, or the Google Cloud CLI.
- Browser
-
-
Open the Google Cloud Console.
-
Go to the Pub/Sub product page.
-
Click CREATE TOPIC on the top of the screen.
-
Fill in the Topic ID field and choose any other options you need.
-
Click CREATE TOPIC in the modal dialog.
You can now use the topic to connect with Akka Serverless
-
- Google Cloud CLI
-
gcloud pubsub topics create TOPIC_ID
You can now use the topic to connect with Akka Serverless