Working with Frameworks

Qualcomm® AI Hub supports the use of multiple Qualcomm® AI Engine Direct versions for compile, profile, and inference jobs.

Querying Frameworks

To programmatically get a list of all supported Qualcomm® AI Engine Direct versions, use get_frameworks().

import qai_hub as hub

supported_frameworks = hub.get_frameworks()
print(supported_frameworks)

Note that this is also availiable through the CLI using

qai-hub list-frameworks

There are two special tags for each framework:

default: Refers to the default version of the framework used in AI Hub.
latest: Refers to the most recently released version of framework used in AI Hub.

Submitting Jobs with Framework Versions

Compile, profile, or inference jobs can be submitted with a specific framework version identified by the api_version listed in get_frameworks()

import qai_hub as hub

compile_job = hub.submit_compile_job(
    model="mobilenet_v2.onnx",
    device=hub.Device("Samsung Galaxy S23 (Family)"),
    target_runime="qnn_context_binary",
    options="--qairt_version 2.31"
)
print(compile_job)

Note that profile jobs that include compiled Qualcomm® AI Engine Direct assets will automatically use the Qualcomm® AI Engine Direct version from the compile job.

The frameworks can also be identified using the tags of default or latest. The following code sample uses the latest availiable version of the Qualcomm® AI Engine Direct framework.

import qai_hub as hub

compile_job = hub.submit_compile_job(
    model="mobilenet_v2.onnx",
    device=hub.Device("Samsung Galaxy S23 (Family)"),
    target_runime="qnn_context_binary",
    options="--qairt_version latest"
)
print(compile_job)

The following code sample uses the default (most stable) version of the Qualcomm® AI Engine Direct framework.

import qai_hub as hub

compile_job = hub.submit_compile_job(
    model="mobilenet_v2.onnx",
    device=hub.Device("Samsung Galaxy S23 (Family)"),
    target_runime="qnn_context_binary",
    options="--qairt_version default"
)
print(compile_job)