kakao developers

REST API

This document describes how to integrate Pose APIs into your service with a REST API. You can use Pose APIs only using a REST API because the Kakao SDK does not support Pose APIs.

• Set the HTTP connection timeout to 5 seconds and the read timeout to 15 seconds when requesting an API.
• If you send too many requests over the request limit per second, an error may occur.
• The response of the Pose API is provided in COCO(Common Objects in Context) data format used for object recognition and detection.
• To see the response codes, refer to REST API Reference.

You can test some features described in this document in [Tools] > [REST API Test].

REST API Test Tooloutlink

Analyze image

The Analyzing image API detects people in the given image and extracts each person's 17 key points(person's eyes, nose, shoulders, elbows, wrists, pelvis, knees, and ankles) to determine their pose.

Add your REST API key to the request header, and send a POST request. You can either upload an image file (file) or specify an image URL (image_url) to request the image analysis.

If the request is successful, the Person object containing the coordinates and confidence score of each key point is returned in JSON format. If there are multiple people in the image, a list of Person objects containing the information of the detected people is returned.

Request

URL

POST /pose HTTP/1.1
Host: cv-api.kakaobrain.com 
Authorization: KakaoAK ${REST_API_KEY} 

Header

Name	Description	Required
Authorization	REST API key. You can check your app's REST API key in [My Application] > [App Keys].	O

Parameter

Name	Type	Description	Required
image_url	String	Image URL.	O*
file	Binary	Image file.	O*

* Either 'image_url' or 'file' is required.

Response

Person

Name	Type	Description
area	Float	Area of the bounding box that includes all key points.
bbox	Float[]	X and Y coordinates of the uppermost key point among the detected key points, and the width(w) and height(h) of the bounding box. [x, y, w, h]
category_id	Int	Fixed as 1. 1: person
keypoints	Float[]	Array containing the coordinates (x, y) and competence (score) of the 17 key points detected in the image in [x_1, y_1, score_1, x_2, y_2, score_2, ..., x_17, y_17, score_17] format. The number from 1 to 17 that comes after x, y, and score refers to the key point ID indicating each body part of the detected person (Refer to Keypoints below.) Example: [X coordinate of the nose, Y coordinate of the nose, confidence of the nose key point, X coordinate of the left eye, Y coordinate of the left eye, confidence of the left eye key point, ... , X coordinate of the right ankle, Y coordinate of the right ankle, confidence of the right ankle key point]
score	Float	Confidence score of each key point. A value between 0 and 1.0.

Keypoints

Key point ID	Body part
1	nose
2	left_eye
3	right_eye
4	left_ear
5	right_ear
6	left_shoulder
7	right_shoulder
8	left_elbow
9	right_elbow
10	left_wrist
11	right_wrist
12	left_hip
13	right_hip
14	left_knee
15	right_knee
16	left_ankle
17	right_ankle

Sample

Request: Using image URL

curl -v -X  POST "https://cv-api.kakaobrain.com/pose" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -H "Authorization: KakaoAK ${REST_API_KEY}" \
  --data-urlencode "image_url=https://example.com/example.jpg" 

Request: Using image file

curl -v -X POST "https://cv-api.kakaobrain.com/pose" \
  -H "Content-Type: multipart/form-data" \
  -H "Authorization: KakaoAK ${REST_API_KEY}" \
  -F "file=@example_pose.jpg" 

Response

[
    {
        "area": 101090.2833,
        "bbox": [719.4526, 244.1255, 182.7314, 553.2178],
        "category_id": 1,
        "keypoints": [
            805.4897, 256.4165, 0.8422, 819.5366, 245.0034, 0.8773, 795.8325, 244.1255, 0.8664, 845.8745, 254.6606, 0.8105, 788.8091, 251.1489, 0.0631, 885.3813, 320.5054, 0.7525, 749.3022, 331.9185, 0.7706, 898.5503, 377.5708, 0.7825, 719.4526, 414.4438, 0.7897, 901.1841, 435.5142, 0.7782, 749.3022, 443.4155, 0.8086, 852.02, 504.8706, 0.6854, 785.2974, 511.894, 0.6738, 833.5835, 644.4614, 0.7899, 800.2222, 659.3862, 0.7655, 833.5835, 796.3433, 0.7055, 824.8042, 743.6675, 0.5165
        ],
        "score": 0.7185
    }
]

Analyze video

The Analyzing video API detects people in each frame of the requested video and extracts key points. Because this API is only used to analyze a given video, you must request the Checking video analysis result API additionally to check the job result by passing the job_id received in this API's response. It is recommended to use the callback function using callback_url to check if the video analysis is completed before requesting the Checking video analysis result API.

Add your REST API key to the request header, and send a POST request. You can either upload a video file(file) or specify a video URL (video_url) to request the video analysis.

If the request is successful, job_id assigned to the analyzed job is returned.

Request

URL

POST /pose/job HTTP/1.1
Host: cv-api.kakaobrain.com 
Authorization: KakaoAK ${REST_API_KEY} 

Header

Name	Description	Required
Authorization	REST API key. You can check your app's REST API key in [My Application] > [App Keys].	O

Parameter

Name	Type	Description	Required
video_url	String	Used to specify a video URL to request the video analysis. HTTP (80 port) and HTTPS (443 port) are supported.	O*
file	Binary	Used to upload a video file to request the video analysis.	O*
smoothing	Boolean	Whether to apply the smoothing process to the position of the key points between the detected frames. true or false (Default: true)	X
callback_url	String	Used to set a callback URL to receive a callback when the video analysis is completed. It is recommended to use HTTPS (443 port). The callback is sent only once in the format of Sample: Callback and not re-sent even when it fails.	X

* Either 'video_url' or 'file' is required.

Response

Name	Type	Description
job_id	String	Job ID of the requested video.

Sample

Request: Using video URL

curl -v -X POST "https://cv-api.kakaobrain.com/pose/job" \
    -H "Content-Type: application/x-www-form-urlencoded" \
    -H "Authorization: KakaoAK ${REST_API_KEY}" \
    --data-urlencode "video_url=http://example.com/example.mp4" 

Request: Using video file

curl -v -X POST "https://cv-api.kakaobrain.com/pose/job" \
    -H "Content-Type: multipart/form-data" \
    -H "Authorization: KakaoAK ${REST_API_KEY}" \ 
    -F "file=@example.mp4" 

Response

{
    "job_id":"bb91c265-341d-4661-813b-870cff0de1d3"
}

Sample: Callback

curl -v -X GET "https://api.example.com/pose/callback?job_id=bb91c265-341d-4661-813b-870cff0de1d3"

Check video analysis result

The Checking video analysis result API returns the processing status and the video analysis results processed through the Analyzing video API.

Add your REST API key to the request header, and send a GET request. To check the result of video analysis, you must add the job_id passed in the response of the Analyzing video API to the request URL.

If the video analysis is successfully completed, the objects containing the information on key points for each person in each frame is returned in JSON format.

Request

URL

GET /pose/job/{job_id} HTTP/1.1
Host: cv-api.kakaobrain.com 
Authorization: KakaoAK ${REST_API_KEY} 

Header

Name	Description	Required
Authorization	REST API key. You can check your app's REST API key in [My Application] > [App Keys].	O

Parameter

Name	Type	Description	Require
job_id	String	Job ID passed in the response of the Analyzing video API.	O

Response

Name	Type	Description
job_id	String	Job ID of the requested video.
status	String	Response status: one of waiting, processing, success, failed, not found - waiting: Server on standy. - processing: Processing video analysis. - success: Successfully processed. - failed: Failed to process. - not found: Cannot find the job ID or the requested video exceeds the seven day storage limit.
annotations	Annotation[]	List of objects containing the coordinates and score of the key points detected in each frame, as an array with the size of the number of frames. Refer to Annotations below. Only returned if the value of status is success.
categories	Category[]	Object containing the information about key points. Refer to Categories below. Only returned if the value of status is success.
info	Info	Object containing information about the analyzed video such as version, creation date, URL, description, etc. Only returned if the value of status is success.
video	Video	Object containing information about the frames of the requested video, such as the number of frames per second, the total number of frames, the video frame size. Only returned if the value of status is success.
description	String	Reason why the request is failed. Only returned if status is failed. Example: "Failed to get video"

Annotation

Name	Type	Description
frame_num	Int	Number of the frames of the requested video. 0 to n-1 (n = number of frames).
objects	Person[]	Object containing the coordinates and score of the detected key point. To see more about parameters, refer to Person in the Analyzing image API.

Category

Name	Type	Description
id	Int	Fixed as 1. 1: person
keypoints	String[]	Array containing the body part names in English for 17 key points. ["nose", "left_eye", "right_eye", "left_ear", "right_ear", "left_shoulder", "right_shoulder", "left_elbow", "right_elbow", "left_wrist", "right_wrist", "left_hip", "right_hip", "left_knee", "right_knee", "left_ankle", "right_ankle"]
name	String	Fixed as person.
skeleton	List<Int[]>	List of array containing the two connected key points. Example: [1, 2] indicates a line connecting the nose and the left ear.
supercategory	String	Fixed as person.

Sample

Request

curl -v -X GET "https://cv-api.kakaobrain.com/pose/job/bb91c265-341d-4661-813b870cff0de1d3" \
  -H "Authorization: KakaoAK ${REST_API_KEY}" 

Response: Success

{
    "annotations": [
       {
        "frame_num": 0,
        "objects": [
            {
            "area": 211350.1765,
            "bbox": [340.11, 22.28, 302.92, 697.72],
            "category_id": 1,
            "keypoints": [517.0, 185.81, 1.0, 524.27, 171.27, 1.0, 517.0, 171.27, 0.86, 560.61, 200.35, 1.0, 0.0, 0.0, 0.0, 582.41, 265.76, 1.0, 473.4, 243.95, 0.97, 596.95, 345.7, 1.0, 407.99, 164.01, 1.0, 524.27, 309.36, 1.0, 371.65, 84.06, 1.0, 546.08, 447.45, 0.87, 480.66, 432.92, 0.88, 531.54, 600.08, 1.0, 480.66, 541.94, 1.0, 524.27, 701.83, 1.0, 473.4, 607.35, 1.0],
            "score": 0.9563
            }
          ]
       } 
           ... // The results after the first frame is omitted. 
       ],
    "categories": [
        {
        "id": 1,
        "keypoints": ["nose", "left_eye", "right_eye", "left_ear", "right_ear", "left_shoulder", "right_shoulder", "left_elbow", "right_elbow", "left_wrist", "right_wrist", "left_hip", "right_hip", "left_knee", "right_knee", "left_ankle", "right_ankle"],
        "name": "person",
        "skeleton": [[1, 2], [1, 3], [2, 3], [2, 4], [3, 5], [4, 6], [5, 7], [6, 7], [6, 8], [6, 12], [7, 9], [7, 13], [8, 10], [9, 11], [12, 13], [14, 12], [15, 13], [16, 14], [17, 15]],
        "supercategory": "person"
        }
    ],
    "info": {
        "contributor": "Kakao Brain Corp.",
        "date_created": "2020/05/26",
        "description": "Human pose estimation result from Kakao Brain",
        "url": "https://www.kakaobrain.com",
        "version": "191227",
        "year": 2020
    },
    "job_id": "bb91c265-341d-4661-813b-870cff0de1d3",
    "status": "success",
    "video": {
        "fps": 29.97,
        "frames": 30,
        "height": 720,
        "width": 1280
    }
}

Response: Fail

{
    "description": "Failed to get video",
    "job_id": "32b1dc9e-16c6-426b-9b06-d1c3e2abdfb4",
    "status": "failed"
}

Samples

Here are code snippets showing how to use Pose APIs.

Using Python

This is an example of implementing Pose APIs using Python. It is recommended to use Python 3.5 or higher.

Analyzing image

import requests
APP_KEY = '${REST_API_KEY}'
IMAGE_URL = 'http://example.com/example.jpg'
IMAGE_FILE_PATH = 'example.jpg'
session = requests.Session()
session.headers.update({'Authorization': 'KakaoAK ' + APP_KEY})

# Requesting with image URL
response = session.post('https://cv-api.kakaobrain.com/pose', data={'image_url': IMAGE_URL})
print(response.status_code, response.json())

# Requesting with image file 
with open(IMAGE_FILE_PATH, 'rb') as f:
    response = session.post('https://cv-api.kakaobrain.com/pose', files=[('file', f)])
    print(response.status_code, response.json())

Analyzing video

import os
import requests

APP_KEY = '${REST_API_KEY}'
VIDEO_URL = 'http://example.com/example.mp4'
VIDEO_FILE_PATH = 'example.mp4'

session = requests.Session()
session.headers.update({'Authorization': 'KakaoAK ' + APP_KEY})

# Requesting with video URL
response = session.post('https://cv-api.kakaobrain.com/pose/job', data={'video_url': VIDEO_URL})
print(response.status_code, response.json())
job_id = response.json()['job_id']

# Requesting with video file  
assert os.path.getsize(VIDEO_FILE_PATH) < 5e7
with open(VIDEO_FILE_PATH, 'rb') as f:
    response = session.post('https://cv-api.kakaobrain.com/pose/job', files=[('file', f)])
    print(response.status_code, response.json())
    job_id = response.json()['job_id']

Checking video analysis result

import requests
APP_KEY = '${REST_API_KEY}'
session = requests.Session()
session.headers.update({'Authorization': 'KakaoAK ' + APP_KEY})
response = session.get('https://cv-api.kakaobrain.com/pose/job/' + job_id)
print(response.status_code, response.json())

Using COCO format

This is an example of visualizing the image and video analysis result using COCO API.

To use COCO API, you need the pycocotools package. Before installing pycocotools, you must install Cython and NumPy first to set dependency.

pip install Cython numpy
pip install matplotlib pycocotools requests pillow opencv-python

Visualizing image results

import cv2
import numpy as np
from matplotlib import pyplot as plt
from pycocotools.coco import COCO
from requests import Session
 
APP_KEY = '${REST_API_KEY}'
session = Session()
session.headers.update({'Authorization': 'KakaoAK ' + APP_KEY})
 
 
def inference(filename):
    with open(filename, 'rb') as f:
        response = session.post('https://cv-api.kakaobrain.com/pose', files={'file': f})
        response.raise_for_status()
        return response.json()
 
 
def visualize(filename, annotations, threshold=0.2):
    # Ignore key points with low confidence
    for annotation in annotations:
        keypoints = np.asarray(annotation['keypoints']).reshape(-1, 3)
        low_confidence = keypoints[:, -1] < threshold
        keypoints[low_confidence, :] = [0, 0, 0]
        annotation['keypoints'] = keypoints.reshape(-1).tolist()
 
    # Visualization using COCO API
    image = cv2.cvtColor(cv2.imread(filename, cv2.IMREAD_COLOR), cv2.COLOR_BGR2RGB)
    plt.imshow(image)
    plt.axis('off')
    coco = COCO()
    coco.dataset = {
        "categories": [
            {
                "supercategory": "person",
                "id": 1,
                "name": "person",
                "keypoints": ["nose", "left_eye", "right_eye", "left_ear", "right_ear", "left_shoulder",
                              "right_shoulder", "left_elbow", "right_elbow", "left_wrist", "right_wrist", "left_hip",
                              "right_hip", "left_knee", "right_knee", "left_ankle", "right_ankle"],
                "skeleton": [[1, 2], [1, 3], [2, 3], [2, 4], [3, 5], [4, 6], [5, 7], [6, 7], [6, 8], [6, 12], [7, 9],
                             [7, 13], [8, 10], [9, 11], [12, 13], [14, 12], [15, 13], [16, 14], [17, 15]]
            }
        ]
    }
    coco.createIndex()
    coco.showAnns(annotations)
    plt.show()
 
 
IMAGE_FILE_PATH = 'example_pose.jpg'
result = inference(IMAGE_FILE_PATH)
visualize(IMAGE_FILE_PATH, result)

Visualizing video results

import os
import time
 
import numpy as np
from matplotlib import pyplot as plt
from pycocotools.coco import COCO
from requests import Session
 
APP_KEY = '${REST_API_KEY}'
 
session = Session()
session.headers.update({'Authorization': 'KakaoAK ' + APP_KEY})
 
 
def submit_job_by_url(video_url):
    response = session.post('https://cv-api.kakaobrain.com/pose/job', data={'video_url': video_url})
    response.raise_for_status()
    return response.json()


def submit_job_by_file(video_file_path):
    assert os.path.getsize(video_file_path) < 5e7
    with open(video_file_path, 'rb') as f:
        response = session.post('https://cv-api.kakaobrain.com/pose/job', files=[('file', f)])
        response.raise_for_status()
        return response.json()


 
# It is recommended to implement a callback when applying it in your real service. 
def get_job_result(job_id):
    while True:
        response = session.get('https://cv-api.kakaobrain.com/pose/job/' + job_id)
        response.raise_for_status()
        response = response.json()
        if response['status'] in {'waiting', 'processing'}:
            time.sleep(10)
        else:
            return response
 
 
def visualize(resp, threshold=0.2):
    # Visualization using COCO API
    coco = COCO()
    coco.dataset = {'categories': resp['categories']}
    coco.createIndex()
    width, height = resp['video']['width'], resp['video']['height']
 
    # Ignore key points with low confidence
    for frame in resp['annotations']:
        for annotation in frame['objects']:
            keypoints = np.asarray(annotation['keypoints']).reshape(-1, 3)
            low_confidence = keypoints[:, -1] < threshold
            keypoints[low_confidence, :] = [0, 0, 0]
            annotation['keypoints'] = keypoints.reshape(-1).tolist()
 
        plt.axis('off')
        plt.title("frame: " + str(frame['frame_num'] + 1))
        plt.xlim(0, width)
        plt.ylim(height, 0)
        coco.showAnns(frame['objects'])
        plt.show()
 

VIDEO_URL = 'http://example.com/example.mp4'
VIDEO_FILE_PATH = 'example.mp4'

# Used when specifying video URL(video_url)
submit_result = submit_job_by_url(VIDEO_URL)
# Used when uploading video file(file)
submit_result = submit_job_by_file(VIDEO_FILE_PATH)

job_id = submit_result['job_id']

job_result = get_job_result(job_id)
if job_result['status'] == 'success':
    visualize(job_result)
else:
    print(job_result)

See more

• Reference for REST API