Inference¶
Session instances encapsulate the entire runtime state for a model. Once you've created a new instance with new and loaded a model, you can access the model's setting keys with the getter (getDouble, getInt, getStream, getString) and setter (setDouble, setInt, setStream, setString) functions. Use setHandler to register callback functions tied to specific events. Call run or push to run model inference.
Callback¶
Session reports events and steps through iterations by calling these user-defined functions or methods.
callback¶
typedef struct SnsrCallback_ *SnsrCallback;
typedef SnsrRC (*SnsrHandler)(SnsrSession s, const char *key, void *data);
typedef void (*SnsrDataRelease)(const void *data);
SNSR_API SnsrCallback
snsrCallback(SnsrHandler h, SnsrDataRelease r, void *data);
Parameters and return value
-
h - The function to call when this callback is invoked.
-
r - A function that releases memory allocated for
data, called when the returned handle is destroyed. UseNULLif there's no clean-up required. -
data - User-defined private value, not used by the SDK.
snsrCallback: A new callback handle instance, orNULLif one could not be created.-
s - The Session handle the callback was registered with.
-
key - Name of the invoked event or iterator setting.
SnsrHandler: OK if all is well. Errors are reported to the code that invoked the callback.
snsrCallback creates a new SnsrCallback instance. These are used to invoke library event handlers and iterators.
- New handles have a reference count of
0. Use retain if keeping a reference and release to release such a reference. - The session will call function
hwhen the callback is invoked. - The allocator will call
rto releasedatajust before theSnsrCallbackis deleted.
Callbacks happen on the same thread that the forEach, push, or run function was called on. Task processing is single-threaded and pauses until the callback function returns. Best practice is to do as little processing as possible in callback handlers.
SnsrHandler instances are user-defined functions that process event callbacks issued by Session.
SnsrDataRelease instances are user-defined functions called to release the data parameter.
The Java language binding uses the SnsrSession.Listener interface for event and iteration callbacks.
Session¶
SnsrSession is the TrulyNatural SDK API inference type. Session handles contain the entire state for a task.
If a SnsrSession handle has to be shared across threads, all calls into this library that use the handle must be protected by application-level thread synchronization calls. The SnsrSession API calls are not re-entrant.
Synchronization and locking is not needed if each instance is accessed from a single thread only.
Best practice is to use a single thread per handle. Using multiple handles per thread is fully supported.
SnsrSession is the TrulyNatural SDK inference class. Session handles contain the entire state for a task.
Instance method calls are not re-entrant. If these are called from different threads, all such calls must be protected by application-level synchronization.
Methods that are not explicitly used to query the state return the instance itself, to support method chaining.
Listener¶
onEvent¶
The C language binding uses Callback for event and iteration handlers.
Parameters and return value
Session reports events and steps through iterations by calling this method.
Callbacks happen on the same thread that the forEach, push, or run method was called on. Task processing is single-threaded and pauses until the onEvent method returns. Best practice is to do as little processing as possible in callback handlers themselves.
Return OK to continue processing, or any other RC value to stop.
Note
SnsrSession.Listener is a functional interface, so you can use a lambda expression instead of passing an object that implements Listener:
clearRC¶
Parameters and return value
-
s - Session handle.
This function is not available in the Java language binding.
describeError¶
Parameters and return value
-
format String.format()style format string.- The same Session instance, for method chaining.
Sets the detailed session error message.
describeError sets the message returned by errorDetail. Use this to propagate a human-readable error message generated by a callback function to the session handle.
dup¶
Parameters and return value
- A duplicate instance of Session.
Duplicates a session handle.
Creates a new session handle that is a clone of a source handle.
This function is roughly equivalent to:
The new session dst contains all of the configuration from the source session, but none of the runtime settings. Immutable configuration settings (such as acoustic models) are shared between the source and dst. Using dup to create two runtime instances of a task will use less memory than loading the same task data into two session handles.
errorDetail¶
Parameters and return value
-
s - Session handle.
- An error message describing the most recent reason for failure. The memory pointed to is owned by this library and most not be released. It is not reference-counted, calling retain or release on this handle will panic the heap allocator. The handle remains valid only until the next function call on
s.
Retrieves a detailed session error message.
This human-readable error message should be used for display or logging only. The content of the message is system-specific. Do not parse this to determine program flow, use the RC return code instead.
forEach¶
Parameters and return value
Parameters and return value
getDouble¶
Parameters and return value
-
key - A key that identifies which setting to read.
- The value of
keyas a double-precision floating-point number.
Retrieves a double value from a session setting.
Use getDouble to read setting keys with double or int values.
getInt¶
Parameters and return value
-
key - A key that identifies which setting to read.
- The value of
keyas an integer.
Retrieves a int value from a session setting.
Use getInt to read setting keys with int or double values.
If the key has a double value that either includes a fractional part, or is too large to fit into a 32-bit int, getInt will return INCORRECT_SETTING_TYPE.
getStream¶
Parameters and return value
Retrieves a stream handle from a runtime session setting or template slot.
If key refers to a slot in a task template, stream will contain a serialized version of that slot in CONFIG format.
The returned stream is valid only until the next library call on the session handle. If you need to extend this, retain the handle and release it when is no longer useful.
Retrieves a stream handle from a runtime session setting or template slot.
If key refers to a slot in a task template, stream will contain a serialized version of that slot in CONFIG format.
getString¶
Parameters and return value
Retrieves a string value from a runtime session setting.
If the key has an int or double value, getString will convert it to a string.
The returned string is valid only until the next library call on the session handle. If you need to extend this, retain the handle and release it when is no longer useful.
load¶
Loads a task model into a Session.
Loads a task or configuration settings from a Stream into the Session. The stream data must be in the format produced by save.
This function retains a reference to inputStream on entry and releases it on exit. load will open inputStream, but will not close it explicitly. Streams are closed before deallocation, so inputStream would be closed if no other references to it are held.
new¶
Parameters and return value
-
s - A pointer to a memory location that will receive the Session handle.
- OK for success, any other value indicates failure. If
sis notNULL, best practice is to retrieve the detailed error message with errorDetail.
Parameters and return value
- A new instance of the SnsrSession class.
This function creates a new Session handle that contains the entire state for a task.
Implementation details
snsrNew() is a preprocessor macro that selects one of three function variants depending whether macros SNSR_OMIT_OSS_COMPONENTS and SNSR_USE_SUBSET are defined.
#ifndef SNSR_USE_SUBSET
# ifdef SNSR_OMIT_OSS_COMPONENTS
# define snsrNew(s) snsrNewOmitOSS((s), SNSR_VERSION)
# else
# define snsrNew(s) snsrNewIncludeOSS((s), SNSR_VERSION)
# endif
#else
# define snsrNew(s) snsrNewSubset((s), SNSR_VERSION)
#endif
SNSR_API SnsrRC
snsrNewSubset(SnsrSession *s, const char *version);
SNSR_API SnsrRC
snsrNewIncludeOSS(SnsrSession *s, const char *version);
SNSR_API SnsrRC
snsrNewOmitOSS(SnsrSession *s, const char *version);
profile¶
pre-release
Writes a human-readable model inference timing profile to a Stream.
Pre-release
This is an experimental feature. Do not use unless recommended by Sensory.
push¶
Parameters and return value
-
s - Session handle.
-
name - The name of an input stream, such as ->audio-pcm.
-
data - Read-only pointer to data, typically 16-bit linear PCM encoded audio in little-endian format.
-
sizeInBytes - The number of bytes of
datato process. - OK for success, any other value indicates failure.
Parameters and return value
-
name - The name of an input stream, such as ->audio-pcm.
-
data - Data to process, typically 16-bit linear PCM encoded audio in little-endian format.
- The same Session instance, for method chaining.
Processes data in a session.
Adds data to the name input Stream for Session s, then processes these and other buffered stream data. Invokes events registered in the session and writes to any output streams the model defines.
Processing continues until one of these is true:
- Everything in
datais processed. - One of the invoked callbacks returns an error code, which
pushwill return. Use INTERRUPTED, REPEAT, SKIP, STOP, or TIMED_OUT to indicate that the operation was stopped on request. - The push-duration-limit timeout is reached.
push invokes event callbacks on the thread that it was called on. The processing loop is single-threaded, so it stalls while waiting for user functions to return.
If push-duration-limit is used, push has to defer processing when the duration limit is reached. In this case, data are added to an internal ring buffer. push report NOT_ENOUGH_SPACE if no space remains for this deferral. To address such a failure, you could:
- Increase the push-buffer-size.
- Increase the push-duration-limit.
- Use a model with lower CPU requirements.
run, stop, push-buffer-backlog, push-buffer-size, push-duration-limit
rC¶
Retrieves the session return code.
Returns the most recent return code from session s.
rCMessage¶
Parameters and return value
-
returnCode - A session or stream return code.
- A string describing the return code. The memory pointed to is owned by this library and must not be released. It is not reference-counted. Calling retain or release on it will panic the heap allocator.
Describes a return code.
Provides a human-readable error message describing returnCode. This message should be used for display or logging only. The content of the message is unspecified and system-specific. Do not parse this to determine program flow, use the RC code instead.
If a Session handle is available, use errorDetail instead, as this provides additional details.
This function is not available in the Java language binding. Use the errorDetail method instead.
release¶
The C language binding uses reference counting to manage object lifetimes, see retain and release in the memory management section.
Releases the native library handle encapsulated by the SnsrSesion class.
This method releases native handles immediately. Use this to free up resources without having to depend on garbage collection cycle eventually running.
No instance methods must be invoked after calling this method.
require¶
Parameters and return value
-
s - Session handle.
-
key - Setting to validate: task-type, task-version, or task-type-and-version-list.
-
value - String to match
keyagainst. - OK for success, any other value indicates failure.
Parameters and return value
-
key - Setting to validate: task-type, task-version, or task-type-and-version-list.
-
value - String to match
keyagainst. - The same Session instance, for method chaining.
Validates task requirements.
Checks that the value for the key, for the task loaded into the current Session, matches the specified value.
Note
Using require is entirely optional. Use it as a sanity check to verify that the model loaded is of the correct type, and that it has a compatible version.
If key is task-type, version should be one of the task type constants defined in values.
If key is task-version, this function uses semantic versioning precedence to determine whether the task version meets the value version requirement.
If key is task-type-and-version-list, version is a list of acceptable task-type and task-version tuples separated by ;.
Version strings can include range specifiers:
- A range is composed of one or more sets, joined by
||. A version matches a range iff every comparator in at least one of the sets is satisfied by the version. - A set is a list of comparators separated by whitespace. A set is satisfied by the intersection of all the comparators it includes.
- A comparator is an operator followed by a version.
- operators include:
=,<,<=,>,>=: range comparisons.~:taskVersion >= value && taskVersion < Mwhere M is the next major version number.^:taskVersion >= value && taskVersion < Mwhere M is the next major version number, skipping over zeros from the left. FortaskVersion >= 1.0.0this is equivalent to the~operator.^0.minor.patchallows only patch updates, e.g.^0.1.1will accept0.1.2but not0.2.0.
- The default behavior when no operator is specified is
^.
task-type, task-version, task-type-and-version-list
Example
reset¶
Resets a session to its initial state.
This call resets the session error code to OK, clears the error detail, discards all pending results, clears internal buffers, and resets session time and sample counts to 0.
reset is most useful to clear the state of a template model, such as tpl-spot-vad, that was stopped through a callback.
Warning
Invoking this function in a callback handler will lead to undefined behavior.
run¶
Processes stream data in a session.
Enters the main session processing loop. This function will process data read from input streams, invoke events registered in the session and write to any output streams the model defines.
Processing continues until one of these is true:
- One of the data streams reach EOF, in which case
runreturns STREAM_END - One of the invoked callbacks returns an error code, which
pushwill return. Use INTERRUPTED, REPEAT, SKIP, STOP, or TIMED_OUT to indicate that the operation was stopped on request.
run invokes event callbacks on the thread that it was called on. The processing loop is single-threaded, so it stalls while waiting for user functions to return.
run will flush the internal recognition pipeline an input stream reaches EOF, unless auto-flush is set to 0.
save¶
Parameters and return value
-
s - Session handle.
-
format - DataFormat controls serialization behavior.
-
outputStream - Stream where
savewrites the serialization to. - OK for success, any other value indicates failure.
public SnsrSession save(SnsrDataFormat format, SnsrStream outputStream);
public SnsrSession save(SnsrDataFormat format, String fileName);
Parameters and return value
-
format - DataFormat controls serialization behavior.
-
outputStream - Stream where
savewrites the serialization to. -
fileName - Path to a file on disk where
saveshould write to. - The same Session instance, for method chaining.
Serializes a session to a stream.
Saves the configuration or runtime settings of the session to a writable output stream.
set¶
Changes session configuration.
This utility function parses the keyValue string for both keys and double, integer, or string values, then updates the session accordingly. Use this to change task configuration from user-supplied configuration strings provided to a command-line application.
keyValue should be of the form "key=value ...", e.g. "delay=30" or "accuracy=0.5 delete-user=hbg". Use double quotes around string values that include whitespace.
The set(key, value) variants infer the setting type from the type of the value argument. These are equivalent to setDouble, setInt, setStream, and setString.
The set(keyValue) variant parses the keyValue string for both keys and double, integer, or string values, then updates the session accordingly. Use this to change task configuration from user-supplied configuration strings provided to a command-line application.
keyValue should be of the form "key=value ...", e.g. "delay=30" or "accuracy=0.5 delete-user=hbg". Use double quotes around string values that include whitespace.
setDouble¶
Sets a session configuration to a double value.
Use setDouble to change configuration keys that require double or int values.
If the key requires an int value and value either includes a fractional part, or is too large to fit into a 32-bit int, setDouble will not change the key and return INCORRECT_SETTING_TYPE.
setHandler¶
Parameters and return value
Sets a session callback handler.
This sets a function to call when the session raises the event specified by key. It replaces the previous handler for key.
Session ignores events without handlers. Best practice is to register handlers only for events that you need to take action on.
Parameters and return value
Sets a session callback handler.
This sets a method to call when the session raises the event specified by key. It replaces the previous handler for key.
Session ignores events without handlers. Best practice is to register handlers only for events that you need to take action on.
setInt¶
setStream¶
Sets streams for a session, loads models into template slots.
- Associates
streamwith the source or sink specified bykey. - If
keyrefers to a slot in a task template this function loads a model fromstreaminto this slot.
load, ->audio-pcm, <-audio-pcm, 0., 1.
setString¶
stop¶
Stops session processing.
When used with pull mode audio processing, this function will cause run to stop and return STOP. The call to stop returns as soon as it has requested the session to stop, it does not wait until run has returned. It is safe to call stop from a different thread than the one run is executing on.
When used with push mode audio processing and push, this function flushes the session processing pipeline, stops any internal processing threads started for the session, and then returns. If the session detects any events while flushing the pipeline it will invoke the callbacks registered for these before stop returns.
Enumerations¶
DataFormat specifies the save serialization format, and RC has the complete list of TrulyNatural SDK return codes.
DataFormat¶
| Name | Description |
|---|---|
CONFIG | recommended All configuration settings. This is the most common use case. |
CONFIG_PRUNED | Prune unused configuration settings, then serialize all remaining ones. An example of unused configuration settings are wake word operating points other than the currently selected operating-point. |
SUBSET_INIT | Custom initialization code for a model subset. Limited initialization reduces overall executable code size by eliding unused modules. To enable, add the generated custom code to the build, and recompile with -DSNSR_USE_SUBSET. |
RUNTIME | Runtime settings, such as enrollments, only. |
SOURCE | All configuration settings, as C source code run from ROM. Optimized to reduce RAM overhead by running from code space. This code targets the same version of the TrulyNatural SDK used to generate it. If you upgrade the SDK, also update any previously generated C source files. tag-identifier, model-name |
SOURCE_RAM | All configuration settings, as C source code loaded into RAM. Optimized for execution speed by loading model data into RAM. This code targets the same version of the TrulyNatural SDK used to generate it. If you upgrade the SDK, also update any previously generated C source files. tag-identifier, model-name |
SOURCE_PRUNED | Prune unused configuration settings, serialize remainder as C source code. This code targets the same version of the TrulyNatural SDK used to generate it. If you upgrade the SDK, also update any previously generated C source files. An example of unused configuration settings are wake word operating points other than the currently selected operating-point. tag-identifier, model-name |
RC¶
SDK return code. OK indicates success, anything else is a failure.
Use errorDetail to obtain a more detailed description of what went wrong.
| Name | Description |
|---|---|
OK | Operation completed successfully. |
EOF | Unexpected end-of-stream encountered. |
ERROR | Operation failed. |
NOT_OPEN | Stream is not open. |
NOT_FOUND | Resource not found. |
NO_MEMORY | Out of heap memory. |
WRONG_MODE | Incorrect stream mode (read or write). |
INTERRUPTED | Interrupted. |
INVALID_ARG | Invalid argument. |
INVALID_MODE | Invalid mode. |
CANNOT_REOPEN | Cannot re-open this stream. |
INVALID_HANDLE | The stream handle is invalid. |
NOT_IMPLEMENTED | Function is not implemented for this stream type. |
DELIM_NOT_FOUND | Delimiter character not encountered. |
FORMAT_NOT_SUPPORTED | Stream format is not supported. |
BUFFER_OVERRUN | Buffer overrun, stream not read from in time |
BUFFER_UNDERRUN | Buffer underrun, stream not written to in time. |
RESERVED_A | Invalid return code. Not used. |
ARG_OUT_OF_RANGE | Argument value is not in the valid range. |
CHANNEL_EXISTS | A channel with this name already exists in the bin. |
CHANNEL_NOT_BUFFERED | This channel is not backed by a circular buffer. |
CHANNEL_NOT_FOUND | Channel not found. |
CHANNELS_NOT_COMPATIBLE | Source and destination channel types are not compatible. |
CONFIGURATION_INCONSISTENT | Element settings are inconsistent. |
CONFIGURATION_MISSING | Element configuration is missing. |
CONNECTION_NOT_FOUND | Connection not found. |
DST_CHANNEL_NOT_FOUND | Destination channel not found. |
DST_ELEMENT_NOT_IN_BIN | Destination element is not in this bin. |
DST_PORT_IN_USE | The destination port is already connected. |
ELEMENT_ID_NOT_KNOWN | Element ID is not known. The model either requires an updated SDK library, or a build with the default snsrNew() initialization. model:ids, SUBSET_INIT |
ELEMENT_INIT_FAILED | Element initialization failed. |
ELEMENT_INIT_INCOMPLETE | Element initialization is incomplete. |
ELEMENT_IS_NOT_A_BIN | Element is not a bin, function is not available. |
ELEMENT_NOT_IN_BIN | Element is not in this bin. |
ELEMENT_NOT_ROOT_BIN | Element is not the root bin, function is not available. |
ELEMENT_REGISTRATION_FAILED | Element registration failed. |
INCORRECT_SETTING_TYPE | Setting is not of the correct type. |
LICENSE_NOT_VALID | License validation failed. |
LICENSE_LIMIT_EXCEEDED | License runtime duration or recognition limit exceeded. |
ITERATION_LIMIT | Push iteration limit exceeded. |
ELEMENT_API_VIOLATION | Element API processing check failed. |
NAME_NOT_UNIQUE | Name is not unique for this bin. |
NOT_ENOUGH_SPACE | Buffer does not have enough free space. |
NOT_INITIALIZED | Element has not been initialized. |
PUSH_FAILED | Push failed. |
PUSH_DURATION_EXCEEDED | Push duration limit exceeded. |
REPEAT | Repeated by callback. |
SETTING_FAILED | Setting could not be applied. |
SETTING_IS_READ_ONLY | Setting is read-only. |
SETTING_NOT_AVAILABLE | Setting is not available in this context. |
SETTING_NOT_FOUND | Setting not found for this element. |
SKIP | Skipped by callback. |
SRC_CHANNEL_NOT_FOUND | Source channel not found. |
SRC_ELEMENT_NOT_IN_BIN | Source element is not in this bin. |
SRC_PORT_IN_USE | The source port is already connected. |
STOP | Stopped by callback. |
STREAM | Stream operation failed. |
STREAM_END | End-of-stream reached. |
UNKNOWN_OBJECT_TYPE | Unknown configuration object type. |
VALUE_NOT_SET | Setting has no configured value. |
CODE_MODEL_TOO_OLD | SnsrCodeModel task is for an older SDK release. Re-convert from the source *.snsr file for this release. |
RESERVED_B | Invalid return code. Not used. |
NO_MODEL | No task model data found. You'll see this error if you try to read or modify a setting before you've loaded a model into a Session. |
REQUIRE_MISMATCH | Task value does not match requirement. |
VERSION_MISMATCH | Task version does not match the requirement. |
LIBRARY_HEADER | The header file version does not match that of the library. VERSION in snsr.h does not match the version the library archive was compiled with. Use the snsr.h shipped with the TrulyNatural 7.6.1 SDK distribution. |
LIBRARY_TOO_OLD | TrulyNatural library is too old. Loading of a new task failed as it requires features that are not available in this release of the library. Please contact Sensory for assistance. load, VERSION |
ALLOCATOR_EXISTS | A heap allocator is already in use. You must call snsrConfig(SNSR_CONFIG_HEAP_SEGMENT, ...) before any other library calls. |
NO_FUNC | A required implementation function is NULL. |
NOT_SUPPORTED | The feature is not supported by the current library configuration. For example: Attempting to use ALLOC_ADD_POOL with a custom allocator that does not support multiple backing store memory pools. |
TIMED_OUT | Operation timed out. |