Why Generative AI Makes Sense For Edge Computing
Carl, the CTO and cofounder at Avassa, is dedicated to developing an edge control plane that application teams will appreciate. In the vast field of artificial intelligence (AI), generative AI has been in the spotlight thanks to OpenAI’s GPT project. However, significant advancements have also been made in other AI fields, particularly object detection and classification, which have found applications ranging from facial recognition systems to industrial automation.
While object detection and classification specialize in processing visual data to identify objects and their locations, generative AI focuses on creating new content or mimicking the characteristics of input data to produce novel outputs. Both of these AI disciplines rely on complex algorithms and large datasets for training but cater to different types of data and tasks.
Commercial products utilizing applied object detection and classification have been around since the mid-70s, with barcode scanners revolutionizing retail and inventory management. Subsequent advancements include optical character recognition systems that convert physical documents into text and camera technologies with face detection capabilities. Industry experts predict that generative AI will be the next significant AI application to impact edge computing.
According to Accenture researchers, enterprises generate massive amounts of data in various locations like branch offices, retail stores, hospitals, and satellites. Analyzing this data in a central data center becomes impractical. By the end of the decade, a substantial amount of critical data will be produced and processed at the edge, where it originates.
Enterprises across different sectors have responded to this trend by moving their computing resources closer to their users, devices, and the data itself as the physical world meets the digital realm. Edge locations facilitate the collection, filtering, and aggregation of local data, making them ideal for generative processes that can generate results to be consumed locally.
Two significant developments have facilitated the adoption of generative AI at the edge. Firstly, publicly available large language models (LLMs) like OpenAI’s ChatGPT or Google’s PaLM and LaMDA possess files of substantial sizes that are impractical for resource-constrained edge environments. To address this issue, there is a growing trend towards smaller, task-specific models. These models are finely tuned to optimize performance for specific tasks or datasets, significantly reducing resource requirements while expanding the range of real-world use cases.
Secondly, generative AI commonly requires hardware acceleration for inferencing processes to ensure speed, efficiency, and handling multiple client requests simultaneously. Historically, this type of hardware has been expensive and energy-inefficient. However, the availability of compact computers with integrated GPUs has made it economically viable to deploy tuned LLMs in far-edge locations.
The applications of generative AI at the edge are rapidly expanding. Voice-assisted shopper suggestions are emerging in retail environments, interactive question-and-answer systems are being implemented for front-of-house staff in restaurants, sentiment analysis and language translation are being utilized in customer feedback contexts, and autonomous decision-making and suggestions are being deployed in warehouse environments.
As generative AI continues to advance technologically and drive business innovation, innovative applications built on this emerging technology stack will proliferate. While some may be short-lived, others will prove to be valuable and viable.
In conclusion, the field of generative AI offers immense potential for edge computing. By leveraging smaller, task-specific models and cost-effective hardware acceleration, enterprises can tap into the benefits of generative AI at the edge, empowering their operations and enhancing user experiences.
Note: This article was written based on a contribution by Carl, CTO, and cofounder at Avassa, and incorporates relevant insights and perspectives from Accenture researchers and industry experts.
