On-device or cloud: the central decision
Where inference runs shapes everything else. On-device models respond instantly, work offline, keep raw photos and audio on the phone and avoid per-request fees, but they are limited by memory, battery and thermal headroom. Cloud models are far more capable and easy to update, at the cost of latency, connectivity needs and ongoing spend. Battery, thermal limits and storage size add constraints that desktop-class experiments never encounter.
Many apps use both. A small on-device model handles detection, wake words or first-pass classification, and a cloud model is called only for harder requests. We prototype each option on real devices, including mid-range Android handsets, because performance on a flagship simulator tells you little about your users. Testing also covers older operating system versions and low memory, which are common among real-world users.
