Skip to content

Protocol Documentation

Table of Contents

Top

ocellus_module_service.proto

License

Copyright 2021 Byte Motion AB

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Summary

The Ocellus system allows for remote modules to be integrated into it's core module system.

This protobuf describes an API for remote modules to bind themselves to Ocellus and therein recieve streaming events and provide streaming responses.

The API communcates over gRPC with protocol buffers which are described in detail here.

The afformentioned link will also provide instructions for building clients for most languages. More details on bidirectional streaming can be found here.

Click here to download the modules service protobuf file

Click here to download the types protobuf file

See a python example here

BindRequest

Bind payload of the initial request registering the remote module with Ocellus

Field Type Label Description
name string The human readible name of the module. Must be unique
states string repeated The states which the module should recieve state events from the stream.
inputs BindRequest.IOData repeated The input data types the module requires. May have several of the same input data types mapped with different names.
outputs BindRequest.IOData repeated Data the module will output, made available to other modules.

BindRequest.IOData

Module input/output data specification

Field Type Label Description
name string Unique name of the input/output data
dataType DataType Data type of the input/output data

DownloadRefRequest

Download reference payload

Field Type Label Description
urn string

DownloadRefResponse

Field Type Label Description
data bytes
pointCloud PointCloud

ModuleData

Bind payload

Field Type Label Description
bindRequest BindRequest
data OutputData

OutputData

Field Type Label Description
intrinsics Intrinsics intrinsics data
contours Contour repeated contours data
camera Camera cameras data
image bytes images data
pointCloud bytes point clouds data
items Item repeated items data
decimal double generic decimal data
integer int32 generic integer data
string string generic string data
errors Error repeated errors, will cause ocellus to go to an error state

PointCloud

Field Type Label Description
width int32
height int32
vertices Vertex repeated Not included in response if "format=depth" included in URN
mat32FC1Data bytes Only included in response if "format=depth" included in URN 32-bit floats encoded as byte array, every pixel is a value in meters

StateEvent

Field Type Label Description
currentState string
previousState string
data Data Maps of inputs mapped with names specified in the bindRequest.inputs[].name fields.
frameId int64

Vertex

Field Type Label Description
x float
y float
z float

OcellusModuleService

Service for remote modules

Method Name Request Type Response Type Description
Bind ModuleData stream StateEvent stream Used to bind a remote module to the Ocellus.

This will make the module available in Ocellus and open up a bidirectional stream emitting StateEvents.

Each time the state changes, if the registered module is interested in this state, it will receive events during the entirety of that state.

Once an event is received, the Ocellus system will block the module execution loop until a response is received from the remote module or a timout/error occurs. | | DownloadRef | DownloadRefRequest | DownloadRefResponse | Used to download reference URNs provided by StateEvents. The data will only be available from after an event has been received and a response is sent. |

Top

ocellus_types.proto

License

Copyright 2021 Byte Motion AB

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Camera

Field Type Label Description
name string name of camera
position Position Position relative scene center point
quaternion Quaternion Quaternion relative scene center point
eulerAngles EulerAngles EulerAngles relative scene center point
intrinsics Intrinsics

Color

The three-channel RGB color model supplemented with a fourth alpha channel.

Field Type Label Description
r float red channel intensity, a number between 0 and 255
g float green channel intensity, a number between 0 and 255
b float blue channel intensity, a number between 0 and 255
a float a number between 0.0 (fully transparent) and 1.0 (fully opaque).

Contour

Contours can be explained simply as a curve joining all the continuous points (along the boundary).

Field Type Label Description
outerContour Points
holeContours Points repeated
area double

Contours

Field Type Label Description
contours Contour repeated

Data

Field Type Label Description
cameras Data.CamerasEntry repeated cameras data
contours Data.ContoursEntry repeated contours data
items Data.ItemsEntry repeated items data
rgbaImages Data.RgbaImagesEntry repeated rgbaImages data, name to urn. Use the DownloadRef rpc to fetch
pointClouds Data.PointCloudsEntry repeated point clouds data, name to urn. Use the DownloadRef rpc to fetch
decimals Data.DecimalsEntry repeated generic double data
integers Data.IntegersEntry repeated generic int32 data
strings Data.StringsEntry repeated generic string data

Data.CamerasEntry

Field Type Label Description
key string
value Camera

Data.ContoursEntry

Field Type Label Description
key string
value Contours

Data.DecimalsEntry

Field Type Label Description
key string
value double

Data.IntegersEntry

Field Type Label Description
key string
value int32

Data.ItemsEntry

Field Type Label Description
key string
value Items

Data.PointCloudsEntry

Field Type Label Description
key string
value string

Data.RgbaImagesEntry

Field Type Label Description
key string
value string

Data.StringsEntry

Field Type Label Description
key string
value string

Dimension

Field Type Label Description
width float width in meters
height float height in meters
depth float depth in meters

Error

Field Type Label Description
code int32 a well defined error code
message string short description of error
description string long description of error (e.g. includes stack trace)

EulerAngles

yaw_pitch_roll

Field Type Label Description
yaw float z-axis in degrees (0-360)
pitch float y-axis in degrees (0-360)
roll float x-axis in degrees (0-360)
local EulerAngles The rotation as Euler angles in degrees relative to the parent transform's rotation.

ExtremePoint

Field Type Label Description
position Position
pixel Point pixel data relative to the 2D canvas calculated with respect to intrinsics

Intrinsics

Field Type Label Description
width int32 number of columns in pixels
height int32 number of rows pixels
ppx float describe the x pixel coordinates of the principal point (center of projection)
ppy float describe the y pixel coordinates of the principal point (center of projection)
fx float describe the focal length of the image, as a multiple of pixel width and height
fy float describe the focal length of the image, as a multiple of pixel width and height
model Intrinsics.Distortion describes which of several supported distortion models was used to calibrate the image
coeffs float repeated provides an array of up to five coefficients describing the distortion model
fov float repeated array of two elements in degrees defining field of view. first element is horizontal, second is vertical

Item

Field Type Label Description
id int64 required can be used to identify the item
name string required human readible item name
parentCamera Camera required camera item position is relative to
weight float estimated weight in kilograms
confidence float value between 0 and 1 how confident the object detection is
color Color rendition color of the item in the Ocellus viewer
deltaTimestamp int64 unix style timestamp in ms
dimension Dimension
position Position required Position relative to the camera that produces this item
quaternion Quaternion required unless eulerAngles is set Quaternion relative to the camera that produces this item
eulerAngles EulerAngles required unless quaternion is set Eular rotation relative to the camera that produces this item
extremePoints Item.ExtremePointsEntry repeated most extreme points of the item's contour
children Item repeated nested items
pathPlan PathPoint repeated
contour Contour
pixel Point pixel data relative to the 2D canvas calculated with respect to intrinsics

Item.ExtremePointsEntry

Field Type Label Description
key string
value ExtremePoint

Items

Field Type Label Description
data Item repeated

PathPoint

Represents a single point in a path plan.

Field Type Label Description
deltaTime int64 time in ms since previous planned path point (e.g. previos PathPoint object in current array)
position Position Position relative to the item that is owner of the PathPlan
eulerAngles EulerAngles EulerAngles relative to the item that is owner of the PathPlan
quaternion Quaternion Quaternion relative to the item that is owner of the PathPlan
pixel Point pixel data relative to the 2D canvas calculated with respect to intrinsics

Point

Describes a single pixel position.

Field Type Label Description
x double Increases from left to right
y double Increases from top to bottom

Points

Field Type Label Description
points Point repeated

Position

Value relative to the camera source's position in meters

Field Type Label Description
x float
y float
z float
local Position Position of the transform relative to the parent transform.

Quaternion

Set of numbers that comprise a four-dimensional vector space

Field Type Label Description
x float
y float
z float
w float
local Quaternion The rotation of the transform relative to the transform rotation of the parent.

DataType

Name Number Description
DATA_TYPE_UNKNOWN 0
DATA_TYPE_CAMERA 1
DATA_TYPE_CONTOURS 2
DATA_TYPE_ITEMS 3
DATA_TYPE_RGBA_IMAGE 4
DATA_TYPE_POINTCLOUD 5
DATA_TYPE_DECIMAL 6
DATA_TYPE_INTEGER 7
DATA_TYPE_STRING 8
DATA_TYPE_ERROR 9

Intrinsics.Distortion

Name Number Description
DISTORTION_NONE 0
DISTORTION_MODIFIED_BROWN_CONRADY 1
DISTORTION_INVERSE_BROWN_CONRADY 2
DISTORTION_FTHETA 3
DISTORTION_BROWN_CONRADY 4

Top

ocellus_state_service.proto

License

Copyright 2021 Byte Motion AB

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Summary

The Ocellus system allows for state manipulation which controls when remote modules receive data.

Click here to download the state service protobuf file

SetStateRequest

Field Type Label Description
state string

SetStateResponse

Field Type Label Description
currentState string
previousState string

OcellusStateService

Service for external state manipulation

Method Name Request Type Response Type Description
SetState SetStateRequest SetStateResponse For internal use only/testing. Used to change Ocellus internal state.

Scalar Value Types

.proto Type Notes C++ Java Python Go C# PHP Ruby
double double double float float64 double float Float
float float float float float32 float float Float
int32 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. int32 int int int32 int integer Bignum or Fixnum (as required)
int64 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. int64 long int/long int64 long integer/string Bignum
uint32 Uses variable-length encoding. uint32 int int/long uint32 uint integer Bignum or Fixnum (as required)
uint64 Uses variable-length encoding. uint64 long int/long uint64 ulong integer/string Bignum or Fixnum (as required)
sint32 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. int32 int int int32 int integer Bignum or Fixnum (as required)
sint64 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. int64 long int/long int64 long integer/string Bignum
fixed32 Always four bytes. More efficient than uint32 if values are often greater than 2^28. uint32 int int uint32 uint integer Bignum or Fixnum (as required)
fixed64 Always eight bytes. More efficient than uint64 if values are often greater than 2^56. uint64 long int/long uint64 ulong integer/string Bignum
sfixed32 Always four bytes. int32 int int int32 int integer Bignum or Fixnum (as required)
sfixed64 Always eight bytes. int64 long int/long int64 long integer/string Bignum
bool bool boolean boolean bool bool boolean TrueClass/FalseClass
string A string must always contain UTF-8 encoded or 7-bit ASCII text. string String str/unicode string string string String (UTF-8)
bytes May contain any arbitrary sequence of bytes. string ByteString str []byte ByteString string String (ASCII-8BIT)