Catalog guides

Messaging

On this page

Structure#

Messaging currently supports Kafka.

Unit	stly resource	Content
Cluster	`messaging-clusters`	A logical Kafka Cluster. Name, handle, summary, description, and environment description
Topic	`kafka-topics`	A Topic in a Cluster. Partition count, retention settings, Message Definitions, and usage notes
Usage note	`kafka-usages`	An application that publishes (`producer`) or consumes (`consumer`) a Topic

Cluster names and handles are unique within an Organization, and Topic names are unique within the same Cluster. For handles, see address names.

Web UI#

Register a Cluster in Queues → 새 Cluster (New Cluster), then add Topics with 새 Topic (New Topic) on the Cluster detail. Record Message Definitions and usage notes together on the Topic editing screen.

Cluster#

{"name": "main-kafka", "handle": "main-kafka", "product": "kafka", "summary": "Kafka Cluster for order events"}

stly messaging-clusters create --input cluster.json

product accepts only kafka.

Topics and Message Definitions#

{
  "name": "order-events",
  "summary": "Order status change events",
  "partition_count": 6,
  "message_definitions": [
    {"name": "OrderStatusChanged", "description": "Published when an order status changes.",
     "key": {"mode": "used", "encoding": "string", "description": "Order ID"},
     "value_encoding": "json", "value_description": "The changed order status",
     "value_schema": "{\"type\":\"object\",\"required\":[\"order_id\",\"status\"],\"properties\":{\"order_id\":{\"type\":\"string\"},\"status\":{\"type\":\"string\"}}}",
     "header_description": "", "external_references": [],
     "examples": [{"name": "Paid", "description": "", "value_json": "{\"order_id\":\"50ECYsM55Ec\",\"status\":\"paid\"}"}]}
  ],
  "usages": [
    {"direction": "producer", "application_name": "orders-service", "description": "Publishes when an order status changes.", "consumer_group": null, "service_id": "1I6Nc76wEPd"}
  ]
}

stly kafka-topics validate --operation create --cluster 6SNzehzO7oI --input topic.json
stly kafka-topics create --cluster 6SNzehzO7oI --input topic.json

value_schema is a JSON Schema source string, used only when value_encoding is json.
An example's value_json is a JSON source string. Write a fictional example for illustration, not a real message.
Encodings are unknown, json, string, binary, avro, protobuf, and other.

Usage notes#

A usage note records an application that publishes or consumes a Topic and its Consumer Group. When you set a Service with service_id, it appears in reverse on the Service detail.

You can add, replace, or delete a single usage note without editing the whole Topic. Use the etag from the Topic detail.

stly kafka-topics get <topic-id>
stly kafka-usages put <usage-id> --topic <topic-id> --input usage.json --if-match '"ETag from the Topic detail"'
stly kafka-usages delete <usage-id> --topic <topic-id> --if-match '"ETag from the Topic detail"'

usage.json takes direction, application_name, description, consumer_group, and an optional service_id. For a new usage note, choose a new ID and use it as <usage-id>.

{"direction": "consumer", "application_name": "billing-service", "description": "Creates invoices from paid orders.", "consumer_group": "billing-order-events"}

stly services kafka-usages <service-id> --fields topic_name,cluster_name,direction

Delete#

A Cluster that still has Topics cannot be deleted (HTTP 409 CLUSTER_IN_USE). Delete its Topics first.