시작하기
설치
Python 버전과 환경을 관리하려면 Miniconda를 사용하는 것이 좋습니다.
- 1단계: Python 환경
컴퓨터에 miniconda 를 설치하세요.
Windows: 설치가 완료되면 시작 메뉴에서 Anaconda Prompt를 엽니다.macOS/Linux: 설치가 완료되면 새 셸 창을 엽니다.Qualcomm® AI Hub 에 대한 환경을 설정하세요:
conda create python=3.10 -n qai_hub conda activate qai_hub
경고
이러한 단계 중 하나라도 SSL: CERTIFICATE_VERIFY_FAILED 오류로 실패하면 SSL 가로채기 및 트래픽 검사 도구가 설치되어 있는 것입니다. IT 부서에 Python pip 및 Python Requests 라이브러리에 대한 인증서를 설정하는 방법에 대한 지침을 요청하세요. 그러면 새 인증서 파일을 제공받을 것입니다. 또한 환경을 만들기 전에 Miniconda에 이 인증서를 추가해야 합니다. conda config --set ssl_verify <path to certificate> 를 사용하여 이 작업을 수행할 수 있습니다.
- 2단계: Python 클라이언트 설치
참고
ARM용 Windows: AI허브 종속성 지원 부족으로 인해 현재 네이티브 ARM Python을 사용할 수 없습니다. Python 생태계가 ARM Windows를 따라잡는 동안, (emulation 을 통해 가능해진) Python의 x64 버전을 설치하세요. Python 다운로드 페이지 에서 Python 3.10 “64비트”를 권장합니다.
pip3 install qai-hub
- 3단계: 로그인
Qualcomm® AI Hub 로 가서 퀄컴 ID로 로그인하면 내가 만든 채용공고에 대한 정보를 볼 수 있습니다.
로그인한 후 Account -> Settings -> API Token 으로 이동하세요. 이를 통해 클라이언트를 구성하는 데 사용할 수 있는 API 토큰을 얻을 수 있습니다.
- Step 4, Option A (recommended): Configure Persistent API Token
Configure the client by persisting the the API token on your workstation. You only need to do this once. This is the recommended way of using AI Hub.
Run the following command in your terminal:
qai-hub configure --api_token INSERT_API_TOKEN
You can check that your API token is installed correctly by fetching a list of available devices. To do that, you can type the following in a Python terminal:
import qai_hub as hub hub.get_devices()
If you go this route, you do not need to follow Option B below.
- Step 4, Option B (for ephemeral environments): Configure Session API Token
In ephemeral customer environments, such as Amazon SageMaker, where API tokens cannot be persistently stored on disk, you will need to configure the AI Hub client on every use.
Provide your API token after you import the AI Hub client. For example:
import qai_hub as hub hub.set_session_token("INSERT_API_TOKEN")
You can check that the API token is working correctly by fetching a list of available devices. Add the following code to your Python script, after initializing the client:
hub.get_devices()
You must provide the API token every time you initialize a session using this token configuration method.
이제 퀄컴 AI허브 에서 작업을 제출할 수 있습니다. 문제가 발생하면 문의하기. 원하는 장치에서 프로필 작업을 실행하려면 아래 예를 자유롭게 시도해 보세요.
간단한 예 (PyTorch)
Qualcomm® AI Hub 환경을 설정한 후 다음 단계는 프로파일링 작업을 제출하는 것입니다. 먼저 이 예제의 종속성을 설치합니다.
pip3 install "qai-hub[torch]"
경고
스니펫 중 하나가 API 인증 오류로 실패하면 유효한 API 토큰이 설치되지 않았다는 의미입니다. 이를 설정하는 방법은 설치 을 참조하세요.
선택한 기기 유형에서 MobileNet v2 네트워크 분석을 제출하세요. 위 목록의 기기를 사용하여 아래 옵션과 다른 기기를 지정할 수 있습니다.
모바일
import numpy as np
import requests
import torch
from PIL import Image
from torchvision.models import mobilenet_v2
import qai_hub as hub
# Using pre-trained MobileNet
torch_model = mobilenet_v2(pretrained=True)
torch_model.eval()
# Step 1: Trace model
input_shape = (1, 3, 224, 224)
example_input = torch.rand(input_shape)
traced_torch_model = torch.jit.trace(torch_model, example_input)
# Step 2: Compile model
compile_job = hub.submit_compile_job(
model=traced_torch_model,
device=hub.Device("Samsung Galaxy S24 (Family)"),
input_specs=dict(image=input_shape),
options="--target_runtime tflite",
)
assert isinstance(compile_job, hub.CompileJob)
# Step 3: Profile on cloud-hosted device
target_model = compile_job.get_target_model()
assert isinstance(target_model, hub.Model)
profile_job = hub.submit_profile_job(
model=target_model,
device=hub.Device("Samsung Galaxy S24 (Family)"),
)
assert isinstance(profile_job, hub.ProfileJob)
# Step 4: Run inference on cloud-hosted device
sample_image_url = (
"https://qaihub-public-assets.s3.us-west-2.amazonaws.com/apidoc/input_image1.jpg"
)
response = requests.get(sample_image_url, stream=True)
response.raw.decode_content = True
image = Image.open(response.raw).resize((224, 224))
input_array = np.expand_dims(
np.transpose(np.array(image, dtype=np.float32) / 255.0, (2, 0, 1)), axis=0
)
# Run inference using the on-device model on the input image
inference_job = hub.submit_inference_job(
model=target_model,
device=hub.Device("Samsung Galaxy S24 (Family)"),
inputs=dict(image=[input_array]),
)
assert isinstance(inference_job, hub.InferenceJob)
on_device_output = inference_job.download_output_data()
assert isinstance(on_device_output, dict)
# Step 5: Post-processing the on-device output
output_name = list(on_device_output.keys())[0]
out = on_device_output[output_name][0]
on_device_probabilities = np.exp(out) / np.sum(np.exp(out), axis=1)
# Read the class labels for imagenet
sample_classes = "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/apidoc/imagenet_classes.txt"
response = requests.get(sample_classes, stream=True)
response.raw.decode_content = True
categories = [str(s.strip()) for s in response.raw]
# Print top five predictions for the on-device model
print("Top-5 On-Device predictions:")
top5_classes = np.argsort(on_device_probabilities[0], axis=0)[-5:]
for c in reversed(top5_classes):
print(f"{c} {categories[c]:20s} {on_device_probabilities[0][c]:>6.1%}")
# Step 6: Download model
target_model = compile_job.get_target_model()
assert isinstance(target_model, hub.Model)
target_model.download("mobilenet_v2.tflite")
IOT
import numpy as np
import requests
import torch
from PIL import Image
from torchvision.models import mobilenet_v2
import qai_hub as hub
# Using pre-trained MobileNet
torch_model = mobilenet_v2(pretrained=True)
torch_model.eval()
# Step 1: Trace model
input_shape = (1, 3, 224, 224)
example_input = torch.rand(input_shape)
traced_torch_model = torch.jit.trace(torch_model, example_input)
# Step 2: Compile model
compile_job = hub.submit_compile_job(
model=traced_torch_model,
device=hub.Device("RB3 Gen 2 (Proxy)"),
input_specs=dict(image=input_shape),
options="--target_runtime tflite",
)
assert isinstance(compile_job, hub.CompileJob)
# Step 3: Profile on cloud-hosted device
target_model = compile_job.get_target_model()
assert isinstance(target_model, hub.Model)
profile_job = hub.submit_profile_job(
model=target_model,
device=hub.Device("RB3 Gen 2 (Proxy)"),
)
assert isinstance(profile_job, hub.ProfileJob)
# Step 4: Run inference on cloud-hosted device
sample_image_url = (
"https://qaihub-public-assets.s3.us-west-2.amazonaws.com/apidoc/input_image1.jpg"
)
response = requests.get(sample_image_url, stream=True)
response.raw.decode_content = True
image = Image.open(response.raw).resize((224, 224))
input_array = np.expand_dims(
np.transpose(np.array(image, dtype=np.float32) / 255.0, (2, 0, 1)), axis=0
)
# Run inference using the on-device model on the input image
inference_job = hub.submit_inference_job(
model=target_model,
device=hub.Device("RB3 Gen 2 (Proxy)"),
inputs=dict(image=[input_array]),
)
assert isinstance(inference_job, hub.InferenceJob)
on_device_output = inference_job.download_output_data()
assert isinstance(on_device_output, dict)
# Step 5: Post-processing the on-device output
output_name = list(on_device_output.keys())[0]
out = on_device_output[output_name][0]
on_device_probabilities = np.exp(out) / np.sum(np.exp(out), axis=1)
# Read the class labels for imagenet
sample_classes = "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/apidoc/imagenet_classes.txt"
response = requests.get(sample_classes, stream=True)
response.raw.decode_content = True
categories = [str(s.strip()) for s in response.raw]
# Print top five predictions for the on-device model
print("Top-5 On-Device predictions:")
top5_classes = np.argsort(on_device_probabilities[0], axis=0)[-5:]
for c in reversed(top5_classes):
print(f"{c} {categories[c]:20s} {on_device_probabilities[0][c]:>6.1%}")
# Step 6: Download model
target_model = compile_job.get_target_model()
assert isinstance(target_model, hub.Model)
target_model.download("mobilenet_v2.tflite")
컴퓨터
import numpy as np
import requests
import torch
from PIL import Image
from torchvision.models import mobilenet_v2
import qai_hub as hub
# Using pre-trained MobileNet
torch_model = mobilenet_v2(pretrained=True)
torch_model.eval()
# Step 1: Trace model
input_shape = (1, 3, 224, 224)
example_input = torch.rand(input_shape)
traced_torch_model = torch.jit.trace(torch_model, example_input)
# Step 2: Compile model
compile_job = hub.submit_compile_job(
model=traced_torch_model,
device=hub.Device("Snapdragon X Elite CRD"),
input_specs=dict(image=input_shape),
options="--target_runtime onnx",
)
assert isinstance(compile_job, hub.CompileJob)
# Step 3: Profile on cloud-hosted device
target_model = compile_job.get_target_model()
assert isinstance(target_model, hub.Model)
profile_job = hub.submit_profile_job(
model=target_model,
device=hub.Device("Snapdragon X Elite CRD"),
)
assert isinstance(profile_job, hub.ProfileJob)
# Step 4: Run inference on cloud-hosted device
sample_image_url = (
"https://qaihub-public-assets.s3.us-west-2.amazonaws.com/apidoc/input_image1.jpg"
)
response = requests.get(sample_image_url, stream=True)
response.raw.decode_content = True
image = Image.open(response.raw).resize((224, 224))
input_array = np.expand_dims(
np.transpose(np.array(image, dtype=np.float32) / 255.0, (2, 0, 1)), axis=0
)
# Run inference using the on-device model on the input image
inference_job = hub.submit_inference_job(
model=target_model,
device=hub.Device("Snapdragon X Elite CRD"),
inputs=dict(image=[input_array]),
)
assert isinstance(inference_job, hub.InferenceJob)
on_device_output = inference_job.download_output_data()
assert isinstance(on_device_output, dict)
# Step 5: Post-processing the on-device output
output_name = list(on_device_output.keys())[0]
out = on_device_output[output_name][0]
on_device_probabilities = np.exp(out) / np.sum(np.exp(out), axis=1)
# Read the class labels for imagenet
sample_classes = "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/apidoc/imagenet_classes.txt"
response = requests.get(sample_classes, stream=True)
response.raw.decode_content = True
categories = [str(s.strip()) for s in response.raw]
# Print top five predictions for the on-device model
print("Top-5 On-Device predictions:")
top5_classes = np.argsort(on_device_probabilities[0], axis=0)[-5:]
for c in reversed(top5_classes):
print(f"{c} {categories[c]:20s} {on_device_probabilities[0][c]:>6.1%}")
# Step 6: Download model
target_model = compile_job.get_target_model()
assert isinstance(target_model, hub.Model)
target_model.download("mobilenet_v2.onnx")
자동차
import numpy as np
import requests
import torch
from PIL import Image
from torchvision.models import mobilenet_v2
import qai_hub as hub
# Using pre-trained MobileNet
torch_model = mobilenet_v2(pretrained=True)
torch_model.eval()
# Step 1: Trace model
input_shape = (1, 3, 224, 224)
example_input = torch.rand(input_shape)
traced_torch_model = torch.jit.trace(torch_model, example_input)
# Step 2: Compile model
compile_job = hub.submit_compile_job(
model=traced_torch_model,
device=hub.Device("SA8650 (Proxy)"),
input_specs=dict(image=input_shape),
options="--target_runtime qnn_lib_aarch64_android",
)
assert isinstance(compile_job, hub.CompileJob)
# Step 3: Profile on cloud-hosted device
target_model = compile_job.get_target_model()
assert isinstance(target_model, hub.Model)
profile_job = hub.submit_profile_job(
model=target_model,
device=hub.Device("SA8650 (Proxy)"),
)
assert isinstance(profile_job, hub.ProfileJob)
# Step 4: Run inference on cloud-hosted device
sample_image_url = (
"https://qaihub-public-assets.s3.us-west-2.amazonaws.com/apidoc/input_image1.jpg"
)
response = requests.get(sample_image_url, stream=True)
response.raw.decode_content = True
image = Image.open(response.raw).resize((224, 224))
input_array = np.expand_dims(
np.transpose(np.array(image, dtype=np.float32) / 255.0, (2, 0, 1)), axis=0
)
# Run inference using the on-device model on the input image
inference_job = hub.submit_inference_job(
model=target_model,
device=hub.Device("SA8650 (Proxy)"),
inputs=dict(image=[input_array]),
)
assert isinstance(inference_job, hub.InferenceJob)
on_device_output = inference_job.download_output_data()
assert isinstance(on_device_output, dict)
# Step 5: Post-processing the on-device output
output_name = list(on_device_output.keys())[0]
out = on_device_output[output_name][0]
on_device_probabilities = np.exp(out) / np.sum(np.exp(out), axis=1)
# Read the class labels for imagenet
sample_classes = "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/apidoc/imagenet_classes.txt"
response = requests.get(sample_classes, stream=True)
response.raw.decode_content = True
categories = [str(s.strip()) for s in response.raw]
# Print top five predictions for the on-device model
print("Top-5 On-Device predictions:")
top5_classes = np.argsort(on_device_probabilities[0], axis=0)[-5:]
for c in reversed(top5_classes):
print(f"{c} {categories[c]:20s} {on_device_probabilities[0][c]:>6.1%}")
# Step 6: Download model
target_model = compile_job.get_target_model()
assert isinstance(target_model, hub.Model)
target_model.download("mobilenet_v2.so")
이 작업은 컴파일 작업을 제출한 후 프로파일 작업을 제출하여 두 작업의 URL을 출력합니다. 마지막으로, 코드는 일부 샘플 데이터에 대해 추론 작업을 수행합니다. 모든 작업은 /jobs/ 에서 확인할 수 있습니다.
작업 상태를 프로그래밍 방식으로 조회할 수 있습니다:
status = profile_job.get_status()
print(status)
아래 스니펫을 사용하여 작업 결과에 액세스할 수 있습니다. 세 가지 주요 부분이 있습니다.
프로필: JSON 형식의 프로파일링 결과입니다.
대상 모델: 배포할 준비가 된 최적화된 모델입니다.
결과: 작업의 모든 아티팩트(로그 포함)가 포함된 폴더.
작업이 완료될 때까지 기다리는 블로킹 API 호출은 다음과 같습니다.
프로파일 결과를 JSON 형식으로 다운로드 (블로킹 호출)
profile = profile_job.download_profile()
print(profile)
최적화된 모델 다운로드 (블로킹 호출)
compile_job.model.download("model_filename")
결과를 현재 디렉토리로 다운로드 (블로킹 호출)
profile_job.download_results(".")
자세한 내용은 모델 프로파일링 로 계속 진행하거나 API documentation 를 참조하세요.