Protocol Documentation

Table of Contents

Top

ocellus_module_service.proto

License

Copyright 2020 Byte Motion AB

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Summary

The Ocellus system allows for remote modules to be integrated into it's core module system.

This protobuf describes an API for remote modules to bind themselves to Ocellus and therein recieve streaming events and provide streaming responses.

The API communcates over gRPC with protocol buffers which are described in detail here.

The afformentioned link will also provide instructions for building clients for most languages. More details on bidirectional streaming can be found here.

Click here to download the protobuf file

See a python example here

BinaryData

Field Type Label Description
data bytes

BindRequest

Bind payload of the initial request registering the remote module with Ocellus

Field Type Label Description
name string The human readible name of the module. Must be unique
states string repeated The states which the module should recieve state events from the stream.
inputs BindRequest.Input repeated The input data types the module requires. May have several of the same input data types mapped with different names.
outputDataTypes BindRequest.DataType repeated Data types the module will output, made available to other modules.

BindRequest.Input

Field Type Label Description
name string Unique name of the input
dataType BindRequest.DataType Data type of the input

Camera

Field Type Label Description
name string name of camera
position Position
quaternion Quaternion
orientation Orientation

Color

The three-channel RGB color model supplemented with a fourth alpha channel.

Field Type Label Description
r float red channel intensity, a number between 0 and 255
g float green channel intensity, a number between 0 and 255
b float blue channel intensity, a number between 0 and 255
a float a number between 0.0 (fully transparent) and 1.0 (fully opaque).

Contours

Contours can be explained simply as a curve joining all the continuous points (along the boundary).

Field Type Label Description
points Point repeated

Dimension

Field Type Label Description
width float width in meters
height float height in meters
depth float depth in meters

DownloadRefRequest

Download reference payload

Field Type Label Description
urn string

InputData

Maps of inputs mapped with names specified in the bindRequest.inputs[].name fields.

Field Type Label Description
intrinsics InputData.IntrinsicsEntry repeated intrinsics data
contours InputData.ContoursEntry repeated contours data
items InputData.ItemsEntry repeated items data
images InputData.ImagesEntry repeated images data, name to urn. Use the DownloadRef rpc to fetch
pointClouds InputData.PointCloudsEntry repeated point clouds data, name to urn. Use the DownloadRef rpc to fetch
pathPlans InputData.PathPlansEntry repeated path plans data
cameras InputData.CamerasEntry repeated cameras data

InputData.CamerasEntry

Field Type Label Description
key string
value Camera

InputData.ContoursEntry

Field Type Label Description
key string
value Contours

InputData.ImagesEntry

Field Type Label Description
key string
value string

InputData.IntrinsicsEntry

Field Type Label Description
key string
value Intrinsics

InputData.ItemsEntry

Field Type Label Description
key string
value Items

InputData.PathPlansEntry

Field Type Label Description
key string
value PathPlans

InputData.PointCloudsEntry

Field Type Label Description
key string
value string

Intrinsics

Field Type Label Description
width int32 number of columns in pixels
height int32 number of rows pixels
ppx float describe the x pixel coordinates of the principal point (center of projection)
ppy float describe the y pixel coordinates of the principal point (center of projection)
fx float describe the focal length of the image, as a multiple of pixel width and height
fy float describe the focal length of the image, as a multiple of pixel width and height
model Intrinsics.Distortion describes which of several supported distortion models was used to calibrate the image
coeffs float repeated provides an array of up to five coefficients describing the distortion model
fov float repeated array of two elements in degrees defining field of view. first element is horizontal, second is vertical

Item

Field Type Label Description
id int64 can be used to identify the item
name string human readible item name
weight float estimated weight in kilograms
confidence float value between 0 and 1 how confident the object detection is
color Color rendition color of the item in the Ocellus viewer
deltaTimestamp int64 unix style timestamp in ms
dimension Dimension
position Position
quaternion Quaternion
orientation Orientation
extremePoints Item.ExtremePointsEntry repeated most extreme points of the item's contour
children Item repeated nested items

Item.ExtremePointsEntry

Field Type Label Description
key string
value Position

Items

Field Type Label Description
data Item repeated

ModuleData

Bind payload

Field Type Label Description
bindRequest BindRequest
data OutputData

Orientation

yaw_pitch_roll

Field Type Label Description
yaw float z-axis in degrees (0-360)
pitch float y-axis in degrees (0-360)
roll float x-axis in degrees (0-360)

OutputData

Field Type Label Description
intrinsics Intrinsics intrinsics data
contours Contours contours data
camera Camera cameras data
image bytes images data
pointCloud bytes point clouds data
items Item repeated items data
pathPlan PathPlan repeated path plans data

PathPlan

Represents a single point in a path plan.

Field Type Label Description
deltaTime int64 time in ms since previous planned path point (e.g. previos PathPlan object in current array)
position Position
orientation Orientation
quaternion Quaternion

PathPlans

Field Type Label Description
data PathPlan repeated

Point

Describes a single pixel position.

Field Type Label Description
x double Increases from left to right
y double Increases from top to bottom

Position

Value relative to the camera source's position in meters

Field Type Label Description
x float
y float
z float

Quaternion

Set of numbers that comprise a four-dimensional vector space

Field Type Label Description
x float
y float
z float
w float

StateEvent

Field Type Label Description
currentState string
previousState string
data InputData
frameId int64

BindRequest.DataType

Name Number Description
DATA_TYPE_ALL 0
DATA_TYPE_INTRINSICS 1
DATA_TYPE_CONTOURS 2
DATA_TYPE_ITEMS 3
DATA_TYPE_IMAGES 4
DATA_TYPE_POINTCLOUD 5
DATA_TYPE_PATH_PLANS 6
DATA_TYPE_CAMERA 7

Intrinsics.Distortion

Name Number Description
DISTORTION_NONE 0
DISTORTION_MODIFIED_BROWN_CONRADY 1
DISTORTION_INVERSE_BROWN_CONRADY 2
DISTORTION_FTHETA 3
DISTORTION_BROWN_CONRADY 4

OcellusModuleService

Service for remote modules

Method Name Request Type Response Type Description
Bind ModuleData stream StateEvent stream Used to bind a remote module to the Ocellus.

This will make the module available in Ocellus and open up a bidirectional stream emitting StateEvents.

Each time the state changes, if the registered module is interested in this state, it will receive events during the entirety of that state.

Once an event is received, the Ocellus system will block the module execution loop until a response is received from the remote module or a timout/error occurs. | | DownloadRef | DownloadRefRequest | BinaryData | Used to download reference URNs provided by StateEvents. The data will only be available from after an event has been received and a response is sent. |

Scalar Value Types

.proto Type Notes C++ Java Python Go C# PHP Ruby
double double double float float64 double float Float
float float float float float32 float float Float
int32 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. int32 int int int32 int integer Bignum or Fixnum (as required)
int64 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. int64 long int/long int64 long integer/string Bignum
uint32 Uses variable-length encoding. uint32 int int/long uint32 uint integer Bignum or Fixnum (as required)
uint64 Uses variable-length encoding. uint64 long int/long uint64 ulong integer/string Bignum or Fixnum (as required)
sint32 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. int32 int int int32 int integer Bignum or Fixnum (as required)
sint64 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. int64 long int/long int64 long integer/string Bignum
fixed32 Always four bytes. More efficient than uint32 if values are often greater than 2^28. uint32 int int uint32 uint integer Bignum or Fixnum (as required)
fixed64 Always eight bytes. More efficient than uint64 if values are often greater than 2^56. uint64 long int/long uint64 ulong integer/string Bignum
sfixed32 Always four bytes. int32 int int int32 int integer Bignum or Fixnum (as required)
sfixed64 Always eight bytes. int64 long int/long int64 long integer/string Bignum
bool bool boolean boolean bool bool boolean TrueClass/FalseClass
string A string must always contain UTF-8 encoded or 7-bit ASCII text. string String str/unicode string string string String (UTF-8)
bytes May contain any arbitrary sequence of bytes. string ByteString str []byte ByteString string String (ASCII-8BIT)