Conway Research Describes Underdog’s On-Device AI and Apple Benchmarks

Conway Research’s Underdog is an AI assistant designed to process tasks on supported devices, with its Apple silicon benchmark offering a specific measure of local performance.

Woof is a 4-billion-parameter model, and Husky is an inference engine built for Apple silicon. Conway’s benchmark reports a peak of 730 tokens per second on an Apple M5 Max for a function-edit task with Flash enabled. For that same task and setup, the reported speed was up to 4.5 times Apple’s MLX framework.

The figures apply to that benchmark. Results varied across the 16 tested tasks, so the peak does not describe performance across all uses or hardware.

Conway describes Underdog as running on Mac and iPhone, including without Wi-Fi, and says user data stays on the device. The product’s documented focus is Apple silicon; its performance on other consumer hardware is not specified.

Underdog is Conway’s first product. The company focuses on AI models that run on devices.