リンク

Link jobs convert one or more Qualcomm® AI Engine Direct DLC models into a single Qualcomm® AI Engine Direct context binary. Link jobs can combine multiple models into a single deployable asset with multiple graphs that share weights. Weights are shared automatically based on name and content. A common use case is to support different instantiations of the same network, such as different input shapes. Partial or no weight sharing is also possible.

注釈

Link jobs are exclusive to Qualcomm® AI Engine Direct DLC models.

入力サイズの変数の例

この例では、モバイルフォンで横向きまたは縦向きの両方で実行できるセグメンテーションネットワークを持っています。このモデルがこれらの入力サイズをサポートするように学習済みと仮定します。

To create this model, we start by compiling two separate DLC models for the two inputs. We then link these two assets together into a single context binary.

This example requires the package Torchvision. First we compile the single-graph DLC models:

import torch
import torchvision

import qai_hub as hub


# Using pre-trained FCN ResNet-50 semantic segmentation model
# We wrap the model in a module to convert the output dictionary containing
# both the primary output and auxiliary output, to return only the primary.
class FCN(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.model = torchvision.models.segmentation.fcn_resnet50(pretrained=True)

    def forward(self, x):
        return self.model(x)["out"]


torch_model = FCN()
torch_model.eval()

input_shape_landscape: tuple[int, ...] = (1, 3, 256, 384)
input_shape_portrait: tuple[int, ...] = (1, 3, 384, 256)

# Trace model

# The traced model will be the same for both input shapes, so the asset can be
# re-used for both compile jobs. For models where different input shapes
# trigger different code paths, link jobs will still work, but both inputs will
# need to be traced separately.
example_input = torch.rand(input_shape_landscape)
pt_model = torch.jit.trace(torch_model, example_input)

src_model = hub.upload_model(pt_model)
device = hub.Device("Samsung Galaxy S24 (Family)")

# Compile both models
compile_job_landscape = hub.submit_compile_job(
    src_model,
    name="FCN Landscape",
    device=device,
    options="--target_runtime qnn_dlc --qnn_options context_enable_graphs=landscape",
    input_specs=dict(image=input_shape_landscape),
)
assert isinstance(compile_job_landscape, hub.CompileJob)

compile_job_portrait = hub.submit_compile_job(
    src_model,
    name="FCN Portrait",
    device=device,
    options="--target_runtime qnn_dlc --qnn_options context_enable_graphs=portrait",
    input_specs=dict(image=input_shape_portrait),
)
assert isinstance(compile_job_portrait, hub.CompileJob)

model_landscape = compile_job_landscape.get_target_model()
model_portrait = compile_job_portrait.get_target_model()
assert isinstance(model_landscape, hub.Model)
assert isinstance(model_portrait, hub.Model)

次に、submit_link_job() を呼び出してモデルをリンクする準備が整いました：

# Link the models
link_job = hub.submit_link_job(
    [model_landscape, model_portrait],
    device=hub.Device("Samsung Galaxy S24 (Family)"),
    name="FCN Landscape+Portrait",
)
assert isinstance(link_job, hub.LinkJob)

linked_model = link_job.get_target_model()
assert isinstance(linked_model, hub.Model)

The resulting model is a Qualcomm® AI Engine Direct context binary that contains multiple graphs. To profile or perform inference on such a model, we need to specify which graph to use with --qnn_options context_enable_graphs=<graph_name>. See Qualcomm® AI Engine Direct Options for more information.

The available graph names in a Qualcomm® AI Engine Direct context binary can be found in the model's Qualcomm® AI Hub page in the Metadata section. In the above example, we deliberately set the graph names during compilation using the --qnn_options context_enable_graphs option. To profile the model above:

# Profile the portrait model
profile_job = hub.submit_profile_job(
    linked_model,
    name="FCN Portrait",
    device=device,
    options="--qnn_options context_enable_graphs=portrait",
)