Glossary What is On-device AI and Next-gen AI?
By null / 1 Jan 2024 / Topics: Generative AI Devices Data and AI
On-device AI refers to local artificial intelligence processes that run on a device without the need for a server or cloud connection. These AI capabilities are also commonly referred to as next-gen, on-device AI. On-device AI workload performance is measured in Tera Operations Per Second (TOPS) and usually exceeds 40 TOPS to support AI workloads from the device. The higher the TOPS measurement, the better the efficiency of the on-device AI capabilities.
Keep in mind that on-device AI is different from AI-capable and AI-ready devices. These devices have been on the market for years — yielding 10 TOPS and 30 TOPS respectively. While on-device AI enables local AI processing without server connectivity, AI-capable devices have AI accelerators on the CPU and GPU.
All three of these terms have become increasingly well-known as more organizations and industries strive toward a new reality powered by AI-related devices and tasks. For example, on-device AI has numerous applications in industries, including:
- Healthcare
- Manufacturing
- Retail
- Finance
- Technology
The applications of on-device AI are transforming processes for many industries, which are supported by the use of AI processor components like Neural Processing Units (NPUs) instead of a costly cloud connection. Businesses that use next-gen, on-device AI will see the added benefits of less latency and inefficiencies and a higher level of local data privacy and security.
However, organizations that rely on outdated devices and hardware may not be able to take advantage of these benefits, as not all devices are capable of supporting AI. As a result, businesses must proactively plan for their next device refresh and ensure their new devices have the proper hardware to accommodate AI-based tasks. This allows organizations to stay ahead of their competition and help teams support business goals fast.
