On-device APIs

Take advantage of machine learning features designed for immediate app integration, with no machine learning experience needed.

What’s new

New APIs in the Vision framework provide advanced image segmentation, animal body pose detection, and 3D human body pose leveraging depth information. Use VisionKit to easily integrate Visual Lookup and subject-lifting experiences into your app. The Natural Language framework enhances understanding of multilingual text using new transformer-based embedding models. The Speech framework makes it easy to introduce custom vocabulary for speech recognition, so you can personalize your user experiences.

Watch the latest video

Vision

Vision

Build features that can process and analyze images and video using computer vision.

View Vision framework

Image Classification

Automatically identify the content in images.

Image Saliency

Quantify and visualize the key part of an image or where in the image people are likely to look.

Image Alignment

Analyze and manage the alignment of images.

Image Similarity

Generate a feature print to compute distance between images.

Object Detection

Find and label objects in images.

Object Tracking

Track moving objects in video.

Trajectory Detection

Detect the trajectory of objects in motion in video.

Contour Detection

Trace the edges of objects and features in images and video.

Text Detection

Detect regions of visible text in images.

Text Recognition

Find, recognize, and extract text from images.

Face Detection

Detect human faces in images.

Face Tracking

Track faces from a camera feed in real time.

Face Landmarks

Find facial features in images by detecting landmarks on faces.

Face Capture Quality

Compare face capture quality in a set of images.

Human Body Detection

Find regions that contain human bodies in images.

Body Pose

Detect landmarks on people in images and video.

Hand Pose

Detect landmarks on human hands in images and video.

Animal Recognition

Find cats and dogs in images.

Barcode Detection

Detect and analyze barcodes in images.

Rectangle Detection

Find rectangular regions in images.

Horizon Detection

Determine the horizon angle in images.

Optical Flow

Analyze the pattern of motion of objects between consecutive video frames.

Person Segmentation New

Produce a matte image for a person in an image.

Document Detection New

Detect rectangular regions in images that contain text.

Natural Language

Natural Language

Analyze natural language text and deduce its language-specific metadata.

View Natural Language framework

Tokenization

Enumerate the words in text strings.

Language Identification

Recognize the language of bodies of text.

Named Entity Recognition

Use a linguistic tagger to name entities in a string.

Part of Speech Tagging

Classify nouns, verbs, adjectives, and other parts of speech in a string.

Word Embedding

Get a vector representation for any word and find similarity between two words or nearest neighbors for a word.

Sentence Embedding

Get a vector representation for any string and find similarity between two strings.

Sentiment Analysis

Score text as positive, negative, or neutral based on the sentiment.

Speech

Speech

Take advantage of speech recognition and saliency features for a variety of languages.

View Speech framework

Speech Recognition

Recognize and analyze speech in audio and get back data like transcripts.

Sound Analysis

Sound Analysis

Analyze audio and recognize it as a particular type, such as laughter or applause.

View Sound Analysis framework

Sound Classification

Analyze sounds in audio using the built-in sound classifier or a custom Core ML sound classification model.