Objects

Definitions

AuthorizationAttribute

An attribute for authorization. This object includes a data type, which represents the type of the attribute attribute (which data types are allowed depends on which authorization plugin is deployed, and how it is configured), and a value. A wildcard can be represented with data type '', and value ''. It means that all authenticated users are allowed to perform an operation.

Name Description Schema
data_type
required		 the type of attribute (e.g., 'team', or 'permission', depending on the Nakadi configuration) string
value
required		 the value of the attribute string

BatchItemResponse

A status corresponding to one individual Event's publishing attempt.

Name Description Schema
detail
optional		 Human readable information about the failure on this item. Items that are not "submitted"
should have a description.		 string
eid
optional		 eid of the corresponding item. Will be absent if missing on the incoming Event. string (uuid)
publishing_status
required		 Indicator of the submission of the Event within a Batch.

- "submitted" indicates successful submission, including commit on he underlying broker.

- "failed" indicates the message submission was not possible and can be resubmitted if so
desired.

- "aborted" indicates that the submission of this item was not attempted any further due
to a failure on another item in the batch.		 enum (submitted, failed, aborted)
step
optional		 Indicator of the step in the publishing process this Event reached.

In Items that "failed" means the step of the failure.

- "none" indicates that nothing was yet attempted for the publishing of this Event. Should
be present only in the case of aborting the publishing during the validation of another
(previous) Event.

- "validating", "partitioning", "enriching" and "publishing" indicate all the
corresponding steps of the publishing process.		 enum (none, validating, partitioning, enriching, publishing)

BusinessEvent

A Business Event.

Usually represents a status transition in a Business process.

Polymorphism : Composition

Name Schema
metadata
required		 EventMetadata

Cursor

Name Description Schema
offset
required		 Offset of the event being pointed to. string
partition
required		 Id of the partition pointed to by this cursor. string

CursorCommitResult

The result of single cursor commit. Holds a cursor itself and a result value.

Name Description Schema
cursor
required		 SubscriptionCursor
result
required		 The result of cursor commit.
- committed: cursor was successfully committed
- outdated: there already was more recent (or the same) cursor committed, so the current one was not
committed as it is outdated		 string

CursorDistanceQuery

Name Schema
final_cursor
required		 Cursor
initial_cursor
required		 Cursor

CursorDistanceResult

Polymorphism : Composition

Name Schema
final_cursor
required		 Cursor
initial_cursor
required		 Cursor

DataChangeEvent

A Data change Event.

Represents a change on a resource. Also contains indicators for the data type and the type of operation performed.

Polymorphism : Composition

Name Description Schema
data
required		 The payload of the type object
data_op
required		 The type of operation executed on the entity.
C: Creation
 U: Update
D: Deletion
 S: Snapshot		 enum (C, U, D, S)
data_type
required		 Example : "pennybags:order" string
metadata
required		 EventMetadata

Event

Note The Event definition will be externalized in future versions of this document.

A basic payload of an Event. The actual schema is dependent on the information configured for the EventType, as is its enforcement (see POST /event-types). Setting of metadata properties are dependent on the configured enrichment as well.

For explanation on default configurations of validation and enrichment, see documentation of EventType.category.

For concrete examples of what will be enforced by Nakadi see the objects BusinessEvent and DataChangeEvent below.

Type : object

EventMetadata

Metadata for this Event.

Contains commons fields for both Business and DataChange Events. Most are enriched by Nakadi upon reception, but they in general MIGHT be set by the client.

Name Description Schema
eid
required		 Identifier of this Event.

Clients MUST generate this value and it SHOULD be guaranteed to be unique from the
perspective of the producer. Consumers MIGHT use this value to assert uniqueness of
reception of the Event.
Example : "105a76d8-db49-4144-ace7-e683e8f4ba46"		 string (uuid)
event_type
optional		 The EventType of this Event. This is enriched by Nakadi on reception of the Event
based on the endpoint where the Producer sent the Event to.

If provided MUST match the endpoint. Failure to do so will cause rejection of the
Event.
Example : "pennybags.payment-business-event"		 string
flow_id
optional		 The flow-id of the producer of this Event. As this is usually a HTTP header, this is
enriched from the header into the metadata by Nakadi to avoid clients having to
explicitly copy this.
Example : "JAh6xH4OQhCJ9PutIV_RYw"		 string
occurred_at
required		 Timestamp of creation of the Event generated by the producer.
Example : "1996-12-19T16:39:57-08:00"		 string (date-time)
parent_eids
optional		 < string (uuid) > array
partition
optional		 Indicates the partition assigned to this Event.

Required to be set by the client if partition strategy of the EventType is
'user_defined'.
Example : "0"		 string
received_at
optional
read-only		 Timestamp of the reception of the Event by Nakadi. This is enriched upon reception of
the Event.
If set by the producer Event will be rejected.
Example : "1996-12-19T16:39:57-08:00"		 string (date-time)
version
optional
read-only		 Version of the schema used for validating this event. This is enriched upon reception.
This string uses semantic versioning, which is better defined in the EventTypeSchema object.		 string

EventStreamBatch

One chunk of events in a stream. A batch consists of an array of Events plus a Cursor pointing to the offset of the last Event in the stream.

The size of the array of Event is limited by the parameters used to initialize a Stream.

If acting as a keep alive message (see GET /event-type/{name}/events) the events array will be omitted.

Sequential batches might present repeated cursors if no new events have arrived.

Name Schema
cursor
required		 Cursor
events
optional		 < Event > array
info
optional		 StreamInfo

EventType

An event type defines the schema and its runtime properties.

Name Description Schema
authorization
optional		 EventTypeAuthorization
category
required		 Defines the category of this EventType.

The value set will influence, if not set otherwise, the default set of
validations, enrichment-strategies, and the effective schema for validation in
the following way:

- undefined: No predefined changes apply. The effective schema for the validation is
exactly the same as the EventTypeSchema.

- data: Events of this category will be DataChangeEvents. The effective schema during
the validation contains metadata, and adds fields data_op and data_type. The
passed EventTypeSchema defines the schema of data.

- business: Events of this category will be BusinessEvents. The effective schema for
validation contains metadata and any additionally defined properties passed in the
EventTypeSchema directly on top level of the Event. If name conflicts arise, creation
of this EventType will be rejected.		 enum (undefined, data, business)
compatibility_mode
optional		 Compatibility mode provides a mean for event owners to evolve their schema, given changes respect the
semantics defined by this field.

It's designed to be flexible enough so that producers can evolve their schemas while not
inadvertently breaking existent consumers.

Once defined, the compatibility mode is fixed, since otherwise it would break a predefined contract,
declared by the producer.

List of compatibility modes:

- 'compatible': Consumers can reliably parse events produced under different versions. Every event published
since the first version is still valid based on the newest schema. When in compatible mode, it's allowed to
add new optional properties and definitions to an existing schema, but no other changes are allowed.
Under this mode, the following json-schema attributes are not supported: not, patternProperties,
additionalProperties and additionalItems. When validating events, additional properties is false.

- 'forward': Compatible schema changes are allowed. It's possible to use the full json schema specification
for defining schemas. Consumers of forward compatible event types can safely read events tagged with the
latest schema version as long as they follow the robustness principle.

- 'none': Any schema modification is accepted, even if it might break existing producers or consumers. When
validating events, no additional properties are accepted unless explicitly stated in the schema.
Default : "forward"		 string
created_at
optional		 Date and time when this event type was created.
Pattern : "date-time"		 string
default_statistic
optional		 EventTypeStatistics
enrichment_strategies
optional		 Determines the enrichment to be performed on an Event upon reception. Enrichment is
performed once upon reception (and after validation) of an Event and is only possible on
fields that are not defined on the incoming Event.

For event types in categories 'business' or 'data' it's mandatory to use
metadata_enrichment strategy. For 'undefined' event types it's not possible to use this
strategy, since metadata field is not required.

See documentation for the write operation for details on behaviour in case of unsuccessful
enrichment.		 < enum (metadata_enrichment) > array
name
required		 Name of this EventType. The name is constrained by a regular expression.

Note: the name can encode the owner/responsible for this EventType and ideally should
follow a common pattern that makes it easy to read and understand, but this level of
structure is not enforced. For example a team name and data type can be used such as
'acme-team.price-change'.
Pattern : "[a-zA-Z][-0-9a-zA-Z_]*(\\.[a-zA-Z][-0-9a-zA-Z_]*)*"
Example : "order.order_cancelled, acme-platform.users"		 string
options
optional		 EventTypeOptions
owning_application
required		 Indicator of the (Stups) Application owning this EventType.
Example : "price-service"		 string
partition_key_fields
optional		 Required when 'partition_resolution_strategy' is set to 'hash'. Must be absent otherwise.
Indicates the fields used for evaluation the partition of Events of this type.

If this is set it MUST be a valid required field as defined in the schema.		 < string > array
partition_strategy
optional		 Determines how the assignment of the event to a partition should be handled.

For details of possible values, see GET /registry/partition-strategies.
Default : "random"		 string
read_scopes
optional		 This field is used for event consuming access control. Nakadi only authorises consumers whose session
contains at least one of the scopes in this list.
If no scopes provided then anyone can consume from this event type.

Usage of read_scopes is deprecated.		 < string > array
schema
required		 EventTypeSchema
updated_at
optional		 Date and time when this event type was last updated.
Pattern : "date-time"		 string
write_scopes
optional		 This field is used for event publishing access control. Nakadi only authorises publishers whose session
contains at least one of the scopes in this list.
If no scopes provided then anyone can publish to this event type.

Usage of write_scopes is deprecated.		 < string > array

EventTypeAuthorization

Authorization section for an event type. This section defines three access control lists: one for producing events ('writers'), one for consuming events ('readers'), and one for administering an event type ('admins'). Regardless of the values of the authorization properties, administrator accounts will always be authorized.

Name Description Schema
admins
required		 An array of subject attributes that are required for updating the event type. Any one of the attributes
defined in this array is sufficient to be authorized. The wildcard item takes precedence over all others,
i.e. if it is present, all users are authorized.		 < AuthorizationAttribute > array
readers
required		 An array of subject attributes that are required for reading events from the event type. Any one of the
attributes defined in this array is sufficient to be authorized. The wildcard item takes precedence over
all others, i.e., if it is present, all users are authorized.		 < AuthorizationAttribute > array
writers
required		 An array of subject attributes that are required for writing events to the event type. Any one of the
attributes defined in this array is sufficient to be authorized.		 < AuthorizationAttribute > array

EventTypeOptions

Additional parameters for tuning internal behavior of Nakadi.

Name Description Schema
retention_time
optional		 Number of milliseconds that Nakadi stores events published to this event type.
Default : 345600000		 integer (int64)

EventTypeSchema

The most recent schema for this EventType. Submitted events will be validated against it.

Name Description Schema
created_at
optional
read-only		 Creation timestamp of the schema. This is generated by Nakadi. It should not be
specified when updating a schema and sending it may result in a client error.
Example : "1996-12-19T16:39:57-08:00"		 string (date-time)
schema
required		 The schema as string in the syntax defined in the field type. Failure to respect the
syntax will fail any operation on an EventType.		 string
type
required		 The type of schema definition. Currently only json_schema (JSON Schema v04) is supported, but in the
future there could be others.		 enum (json_schema)
version
optional
read-only		 This field is automatically generated by Nakadi. Values are based on semantic versioning. Changes to title
or description are considered PATCH level changes. Adding new optional fields is considered a MINOR level
change. All other changes are considered MAJOR level.
Default : "1.0.0"		 string

EventTypeStatistics

Operational statistics for an EventType. This data may be provided by users on Event Type creation. Nakadi uses this object in order to provide an optimal number of partitions from a throughput perspective.

Name Description Schema
message_size
required		 Average message size for each Event of this EventType. Includes in the count the whole serialized
form of the event, including metadata.
Measured in bytes.		 integer
messages_per_minute
required		 Write rate for events of this EventType. This rate encompasses all producers of this
EventType for a Nakadi cluster.

Measured in event count per minute.		 integer
read_parallelism
required		 Amount of parallel readers (consumers) to this EventType. integer
write_parallelism
required		 Amount of parallel writers (producers) to this EventType. integer

Feature

Feature of Nakadi to be enabled or disabled

Name Schema
enabled
required		 boolean
feature
required		 string

Metrics

Object containing application metrics.

Type : object

URI identifying another page of items

Name Description Schema
href
optional		 Example : "/subscriptions?offset=20&limit=10" string (uri)

contains links to previous and next pages of items

Name Schema
next
optional		 PaginationLink
prev
optional		 PaginationLink

Partition

Partition information. Can be helpful when trying to start a stream using an unmanaged API.

This information is not related to the state of the consumer clients.

Name Description Schema
newest_available_offset
required		 An offset of the newest available Event in that partition. This value will be changing
upon reception of new events for this partition by Nakadi.

This value can be used to construct a cursor when opening streams (see
GET /event-type/{name}/events for details).

Might assume the special name BEGIN, meaning a pointer to the offset of the oldest
available event in the partition.		 string
oldest_available_offset
required		 An offset of the oldest available Event in that partition. This value will be changing
upon removal of Events from the partition by the background archiving/cleanup mechanism.		 string
partition
required		 string
unconsumed_events
optional		 Approximate number of events unconsumed by the client. This is also known as consumer lag and is used for
monitoring purposes by consumers interested in keeping an eye on the number of unconsumed events.		 number (int64)

Problem

Name Description Schema
detail
optional		 A human readable explanation specific to this occurrence of the problem.
Example : "Connection to database timed out"		 string
instance
optional		 An absolute URI that identifies the specific occurrence of the problem.
It may or may not yield further information if dereferenced.		 string (uri)
status
required		 The HTTP status code generated by the origin server for this occurrence of the problem.
Example : 503		 integer (int32)
title
required		 A short, summary of the problem type. Written in English and readable for engineers
(usually not suited for non technical stakeholders and not localized)
Example : "Service Unavailable"		 string
type
required		 An absolute URI that identifies the problem type. When dereferenced, it SHOULD provide
human-readable API documentation for the problem type (e.g., using HTML). This Problem
object is the same as provided by https://github.com/zalando/problem
Example : "http://httpstatus.es/503"		 string (uri)

ShiftedCursor

Polymorphism : Composition

Name Description Schema
offset
required		 Offset of the event being pointed to. string
partition
required		 Id of the partition pointed to by this cursor. string
shift
required		 This number is a modifier for the offset. It moves the cursor forward or backwards by the number of events
provided.
For example, suppose a user wants to read events starting 100 positions before offset
"001-000D-0000000000000009A8", it's possible to specify shift with -100 and Nakadi will make the
necessary calculations to move the cursor backwards relatively to the given offset.

Users should use this feature only for debugging purposes. Users should favor using cursors provided in
batches when streaming from Nakadi. Navigating in the stream using shifts is provided only for
debugging purposes.		 number (int64)

Storage

A storage backend.

Name Description Schema
id
required		 The ID of the storage backend.
Maximal length : 36		 string
kafka_configuration
required		 configuration settings for kafka storage. Only necessary if the storage type is 'kafka' kafka_configuration
storage_type
required		 the type of storage. Possible values: ['kafka'] string

kafka_configuration

Name Description Schema
exhibitor_address
optional		 the Zookeeper address string
exhibitor_port
optional		 the Zookeeper path string
zk_address
optional		 the Zookeeper address string
zk_path
optional		 the Zookeeper path string

StreamInfo

This object contains general information about the stream. Used only for debugging purposes. We recommend logging this object in order to solve connection issues. Clients should not parse this structure.

Type : object

Subscription

Subscription is a high level consumption unit. Subscriptions allow applications to easily scale the number of clients by managing consumed event offsets and distributing load between instances. The key properties that identify subscription are 'owning_application', 'event_types' and 'consumer_group'. It's not possible to have two different subscriptions with these properties being the same.

Name Description Schema
consumer_group
optional		 The value describing the use case of this subscription.
In general that is an additional identifier used to differ subscriptions having the same
owning_application and event_types.
Default : "default"
Minimum length : 1
Example : "read-product-updates"		 string
created_at
optional
read-only		 Timestamp of creation of the subscription. This is generated by Nakadi. It should not be
specified when creating subscription and sending it may result in a client error.
Example : "1996-12-19T16:39:57-08:00"		 string (date-time)
event_types
required		 EventTypes to subscribe to.
The order is not important. Subscriptions that differ only by the order of EventTypes will be
considered the same and will have the same id. The size of event_types list is limited by total number
of partitions within these event types. Default limit for partition count is 100.		 < string > array
id
optional
read-only		 Id of subscription that was created. Is generated by Nakadi, should not be specified when creating
subscription.		 string
initial_cursors
optional		 List of cursors to start reading from. This property is required when read_from = cursors.
The initial cursors should cover all partitions of subscription.		 < SubscriptionCursorWithoutToken > array
owning_application
required		 The id of application owning the subscription.
Minimum length : 1
Example : "gizig"		 string
read_from
optional		 Position to start reading events from. Currently supported values:
- begin - read from the oldest available event.
- end - read from the most recent offset.
- cursors - read from cursors provided in initial_cursors property.
Applied when the client starts reading from a subscription.
Default : "end"		 string

SubscriptionCursor

Polymorphism : Composition

Name Description Schema
cursor_token
required		 An opaque value defined by the server. string
event_type
required		 The name of the event type this partition's events belong to. string
offset
required		 Offset of the event being pointed to. string
partition
required		 Id of the partition pointed to by this cursor. string

SubscriptionCursorWithoutToken

Polymorphism : Composition

Name Description Schema
event_type
required		 The name of the event type this partition's events belong to. string
offset
required		 Offset of the event being pointed to. string
partition
required		 Id of the partition pointed to by this cursor. string

SubscriptionEventStreamBatch

Analogue to EventStreamBatch but used for high level streamming. It includes specific cursors for committing in the high level API.

Name Schema
cursor
required		 SubscriptionCursor
events
optional		 < Event > array
info
optional		 StreamInfo

SubscriptionEventTypeStats

statistics of one event-type within a context of subscription

Name Description Schema
event_type
required		 event-type name string
partitions
required		 statistics of partitions of this event-type < partitions > array

partitions

Name Description Schema
partition
required		 string
state
required		 The state of this partition in current subscription. Currently following values are possible:
- unassigned: the partition is currently not assigned to any client;
- reassigning: the partition is currently reasssigning from one client to another;
- assigned: the partition is assigned to a client.		 string
stream_id
optional		 the id of the stream that consumes data from this partition string
unconsumed_events
optional		 The amount of events in this partition that are not yet consumed within this subscription.
The property may be absent at the moment when no events were yet consumed from the partition in this
subscription (In case of read_from is BEGIN or END)		 number