Cloudflare, a leading content delivery network and cloud security platform, is taking a major step forward in democratizing artificial intelligence (AI) by making it accessible to developers. By adding GPU-powered infrastructure and model-serving capabilities to its edge network, Cloudflare is bringing state-of-the-art foundation models to the masses, enabling any developer to tap into its AI platform with a simple REST API call.
Cloudflare originally introduced Workers, a serverless compute platform at the edge, in 2017. This innovative platform allows developers to create JavaScript Service Workers that run directly in Cloudflare’s edge locations worldwide. With Workers, developers can modify a site’s HTTP requests and responses, make parallel requests, and respond directly from the edge. Cloudflare Workers operates on an API that is analogous to the W3C Service Workers standard.
Driven by the rise of generative AI, Cloudflare decided to augment its Workers platform with AI capabilities. As a result, the company has launched three key elements to support AI inference:
1. Strategic Partnerships: Cloudflare has collaborated with tech giants such as NVIDIA, Microsoft, Hugging Face, Databricks, and Meta (formerly Facebook) to bring GPU infrastructure and foundation models to its edge network. Additionally, Cloudflare hosts embedding models that convert text to vectors. The Vectorize database allows for the storage, indexing, and querying of these vectors, providing contextual information to the foundation models and reducing any potential inaccuracies in responses. Cloudflare’s AI Gateway offers observability, rate limiting, and caching of frequent queries, optimizing application performance while reducing costs.
2. Extensive Model Catalog: The Workers AI model catalog offers the latest and most advanced foundation models. From Meta’s Llama 2 to Stable Diffusion XL and Mistral 7B, developers have access to a wide range of models to power their modern AI-driven applications.
3. ONNX Runtime Optimization: To ensure efficient operation of models in resource-constrained environments, Cloudflare utilizes the ONNX Runtime, an open neural network exchange runtime developed by Microsoft. This optimization technology is also used by Microsoft for running foundation models in Windows.
What sets Cloudflare apart is its commitment to providing flexibility to developers. While developers can write AI inference code in JavaScript and deploy it to Cloudflare’s edge network, the models can also be invoked through a simple REST API using any programming language. This versatility allows developers to seamlessly incorporate generative AI into web, desktop, and mobile applications that operate in diverse environments.
In September 2023, Cloudflare initially introduced Workers AI with inference capabilities in seven cities. However, the company has set an ambitious goal to support Workers AI inference in 100 cities by the end of this year, with further plans for near-ubiquitous coverage by the end of 2024.
Cloudflare stands as one of the pioneering content delivery network and edge network providers to integrate AI capabilities into its edge network through GPU-powered Workers AI, vector database, and an AI Gateway for streamlined AI deployment management. Collaborating with industry leaders like Meta and Microsoft, Cloudflare offers an extensive model catalog and leverages the ONNX Runtime for optimal performance.
With the future of AI firmly at the edge, Cloudflare is leading the way in empowering developers and making AI accessible to all. By combining cutting-edge technology with strategic partnerships and a developer-first approach, Cloudflare is democratizing AI and driving innovation in the industry.
