Working with Frameworks

Qualcomm® AI Hub supports the use of multiple Qualcomm® AI Engine Direct versions for compile, profile, and inference jobs.

Querying Frameworks

To programmatically get a list of all supported Qualcomm® AI Engine Direct versions, use get_frameworks().

import qai_hub as hub

supported_frameworks = hub.get_frameworks()
print(supported_frameworks)

Note that this is also availiable through the CLI using

qai-hub list-frameworks

There are two special tags for each framework:

default: Refers to the default version of the framework used in AI Hub.
latest: Refers to the most recently released version of framework used in AI Hub.

Submitting Jobs with Framework Versions

Compile, profile, or inference jobs can be submitted with a specific framework version identified by the api_version listed in get_frameworks()

import qai_hub as hub

compile_job = hub.submit_compile_job(
    model="mobilenet_v2.onnx",
    device=hub.Device("Samsung Galaxy S23 (Family)"),
    target_runime="qnn_context_binary",
    options="--qairt_version 2.31"
)
print(compile_job)

The frameworks can also be identified using the tags of default or latest. The following code sample uses the latest availiable version of the Qualcomm® AI Engine Direct framework.

import qai_hub as hub

compile_job = hub.submit_compile_job(
    model="mobilenet_v2.onnx",
    device=hub.Device("Samsung Galaxy S23 (Family)"),
    target_runime="qnn_context_binary",
    options="--qairt_version latest"
)
print(compile_job)

The following code sample uses the default (most stable) version of the Qualcomm® AI Engine Direct framework.

import qai_hub as hub

compile_job = hub.submit_compile_job(
    model="mobilenet_v2.onnx",
    device=hub.Device("Samsung Galaxy S23 (Family)"),
    target_runime="qnn_context_binary",
    options="--qairt_version default"
)
print(compile_job)

Version Selection

When submitting a profile or inference job with an explicit Qualcomm® AI Runtime version, AI Hub will use that version if it is compatible with the input model; AI Hub will return an error if the requested version is not compatible with the input model.

When submitting a profile or inference job without an explicit Qualcomm® AI Runtime version, AI Hub will automatcally select a compatible version based on the input model.

If the input model is not a Qualcomm® AI Engine Direct asset (i.e. a TensorFlow Lite model, or a standard ONNX model), AI Hub will always select the default Qualcomm® AI Engine Direct version.

If the input model is a DLC or a context binary model Qualcomm® AI Engine Direct asset, AI Hub will query the Qualcomm® AI Engine Direct version from the asset in order to select a compatible Qualcomm® AI Engine Direct version. The default Qualcomm® AI Engine Direct version is used if it is compatible with the model; the latest Qualcomm® AI Engine Direct version is used if the default version is not compatible with the model. An error will be returned if no compatible version is available. If the input model is a model library Qualcomm® AI Engine Direct asset, AI Hub will select the default version and return an error if that is incompatible with the model version. It is recommended to always pass an explicit qairt version when profiling a model library Qualcomm® AI Engine Direct asset.