qai_hub.submit_compile_and_profile_jobs

submit_compile_and_profile_jobs(model, device, name=None, input_specs=None, compile_options='', profile_options='', single_compile=True, calibration_data=None, retry=True)

Submits a compilation job and a profile job.

Parameters:

  • model (Union[Model, TopLevelTracedModule, MLModel, ModelProto, bytes, str, Path]) – Model to profile.

  • device (Union[Device, List[Device]]) – Devices on which to run the jobs.

  • name (Optional[str]) – Optional name for both the jobs. Job names need not be unique.

  • input_specs (Optional[Mapping[str, Union[Tuple[int, ...], Tuple[Tuple[int, ...], str]]]]) –

    Required if model is a PyTorch model. Keys in Dict (which is ordered in Python 3.7+) define the input names for the target model (e.g., Core ML model) created from this profile job, and may be different from the names in PyTorch model.

    An input shape can either be a Tuple[int, …], ie (1, 2, 3), or it can be a Tuple[Tuple[int, …], str], ie ((1, 2, 3), “int32”)). The latter form can be used to specify the type of the input. If a type is not specified, it defaults to “float32”. Currently, only “float32”, “int8”, “int16”, “int32”, “uint8”, and “uint16” are accepted types.

    For example, a PyTorch module with forward(self, x, y) may have input_specs=dict(a=(1,2), b=(1, 3)). When using the resulting target model (e.g. a Core ML model) from this profile job, the inputs must have keys a and b, not x and y. Similarly, if this target model is used in an inference job (see qai_hub.submit_inference_job()), the dataset must have entries a, b in this order, not x, y

    If model is an ONNX model, input_specs are optional. input_specs can be used to overwrite the model’s input names and the dynamic extents for the input shapes. If input_specs is not None, it must be compatible with the model, or the server will return an error.

  • single_compile (bool) – If True, create a single job on a single device compatible with all devices. If False, create a single job for each device

  • compile_options (str) – Cli-like flag options for the compile job. See Compile Options.

  • profile_options (str) – Cli-like flag options for the profile job. See Profile Options.

  • calibration_data (Union[Dataset, Mapping[str, List[ndarray]], str, None]) – Data, Dataset, or Dataset ID to use for post-training quantization. PTQ will be applied to the model during translation.

  • retry (bool) – If job creation fails due to rate-limiting, keep retrying periodically until creation succeeds.

Returns:

jobs – Returns a tuple of CompileJob and ProfileJob.

Return type:

Tuple[CompileJob | None, ProfileJob | None] | List[Tuple[CompileJob | None, ProfileJob | None]

Examples

Submit a traced Torch model for profiling as a QNN Model Library on a Samsung Galaxy S23:

import qai_hub as hub
import torch

pt_model = torch.jit.load("mobilenet.pt")

input_shapes = (1, 3, 224, 224)

model = hub.upload_model(pt_model)

jobs = hub.submit_compile_and_profile_jobs(
    model, device=hub.Device("Samsung Galaxy 23"),
    name="mobilenet (1, 3, 224, 224)",
    input_specs=dict(x=input_shapes),
    compile_options="--target_runtime qnn_lib_aarch64_android"
)

For more examples, see Compiling Models and ref:profile_examples.