One of the most exciting frontiers in a world where technological advancements are reshaping industries at an unprecedented rate is the integration of generative artificial intelligence into enterprise operations.
As organizations strive to streamline processes, elevate customer experiences, and discover new horizons of innovation, the adoption of generative AI has opened up significant potential. Amid this excitement and potential, however, challenges loom large on the path to realizing the full potential of gen AI in enterprise applications.
The capability that we got the most excited about was the ability to follow instructions, said Arjun Prakash, co-founder, and chief executive officer of Distyl AI Inc. When InstructGPT came out … there was this aha moment where we realized that this wasn’t just something that you could use to write letters or edit emails … it’s also something you can use to give instructions and actually carry out tasks that could have a meaningful operational impact at enterprises.
Prakash was joined by Jerry Liu, co-founder, and CEO of LlamaIndex Inc., as they spoke about the challenges and opportunities in generative AI for enterprise applications at the Supercloud 5: The Battle for AI Supremacy event.
In the rapidly evolving landscape of AI, companies are increasingly looking to harness the power of AI to streamline operations, enhance customer experiences, and unlock new opportunities. However, there are significant challenges that need to be overcome to fully realize the potential of generative AI in enterprise deployment, according to Liu.
A lot of people are trying to build LM applications these days, mostly to build prototypes, and they’re finding it hard to productionize, he said. There’s a few kind of core issues. One is hallucination … it might not actually understand some of the outputs. The other piece is that a lot of people are building software systems around outlines, and they’re still figuring out the best practices for doing so.
One of the key takeaways was the concept of retrieval augmented generation, or RAG, which involves combining a knowledge base with a language model, enabling more efficient and accurate information retrieval. RAG is an area where significant progress is being made, with growing enterprise adoption. However, it is not without its challenges, primarily because of the need to carefully handle parameters and data at various stages of the process, according to Liu.
This is exactly where the point about adding more parameters to the system comes in, because the moment you build retrieval, in addition to the language model, you have to think about how does your retrieval system work, he said. How do you load in data … then how do you figure out how to retrieve it? A lot of failure points aren’t just due to the outline. It’s due to the selection of parameters at the earlier stages of the process.
One of the key debates in the ever-evolving landscape of generative AI centers around the concept of fine-tuning — modifying pre-trained language models for specific tasks or domains. Fine-tuning has gained significant attention and discussion within the AI community.
What we have found is that the information loss from fine-tuning is larger than the accuracy gains from treating it as an information retrieval problem outside of the large language model itself, Prakash said. We have really good techniques of doing high reliability and predictable information retrieval. It’s going to be a work in progress to get fine-tuning to the point where you can trust it for information as well.
However, organizations are often caught in a perpetual cycle of trying to match the capabilities of the next AI model iteration. This raises questions about the long-term viability of fine-tuning as AI models continue to advance rapidly.
Fine-tuning may be a temporary solution as AI models become more powerful and costs decrease, according to Liu. The trajectory of AI capabilities suggests that the need for fine-tuning may diminish over time, aligning with an exponential growth curve in model capabilities.
A lot of people are doing fine-tuning, he said. The reason is that, with the current set of models, it allows you to squeeze out better performance for less cost. So, there are certain types of tasks that are very specialized, and you can definitely fine-tune something much smaller and much cheaper versus just using GPT-4 and GPT-3.5.
Generative AI holds immense promise for enterprise applications, but it also presents challenges that need to be addressed. As researchers and industry leaders continue to explore and innovate, the possibilities for leveraging generative AI in the enterprise continue to expand. The journey toward harnessing the full potential of generative AI is underway, and the benefits for businesses and their customers are set to be profound.
