"Moat is another term for a sustainable competitive advantage. It is a startup's ability to maintain a competitive advantage for protecting its market share and long-term profits from its competitors.”The typical references to “LLM economics” or “prompt economics” are monetary: How much will the necessary resources (human, compute, cloud, and associated deployment outlays) affect the bottom line and when will the organization begin to see a return on investment (ROI)? There are plenty of articles, blogs posts, interviews, and opinion pieces out there that offer quantitative insights into all of those and other finance-related issues. This blog post isn't one of them. This post considers the age-old dilemma of balancing the initial outlay vs the cost of waiting in qualitative terms, including operational effects and non-monetary ROI. It identifies the issue behind the dollar signs: What is the real cost of implementing generative artificial intelligence (GenAI) tools, such as large language models (LLMs) vs the opportunity cost of not implementing them? In other words, if it’s too expensive to deploy LLMs now, or too complicated, or too messy, there are easy, almost rote arguments that support the delay: We need the right people in place to (deploy, train, maintain, etc.--take your pick), we need more money on the balance sheet, we need to complete Project X first, and so on. Certainly waiting for a target date down the line, the occurrence of a long-planned corporate event, a market action, or a fireball crossing the sky at midnight will allow for lower costs later. But keep in mind that the term “lower costs” won’t apply just to dollars, but to:
- Diminished competitive advantage, if your competition is using AI
- Ponderous operations as your teams are still executing at human speed instead of at the speed of AI
- Decreased relevance in the marketplace as customers realize you’re stagnating
- Loss of human talent as key people depart for firms offering them the opportunity to learn new skills and work with new technology
Baby Steps
As democratization continues to change the face of AI, deployment, even deployment across the enterprise, can be executed incrementally if the organization has identified its needs. For example:- Have the probable users and utility been identified, such as who gets to use it, how often, and for what purpose?
- Has aligning the model to core business objectives been discussed and decided?
- Have the risks and benefits of open-source vs private models been discussed and decided?
- Is there a hierarchy for key qualities, such as scalability, performance/response quality, latency/speed, and privacy/confidentiality?
- Will the model be used for text-based tasks, such as content generation, summarization, or translation, or will it be a chatbot that must interact with users via voice or text?
- Will the model be unimodal or multimodal?
- How many models are needed?
- Must the model(s) be trained on specific types of data, for instance medical, financial, pharmacological, scientific, etc.?
- What size context window is required to optimize usage?
- Must the activities comply with industry standards or government regulations, such as privacy rules or security requirements?
- What sort of infrastructure is needed to support model use? Will the model(s) be on-premise, cloud-based offerings via AI-as-a-service (AIaaS), software-as-a-service (SaaS), subscription, or via API, or so large a data center is required?
- Are GPUs required for processing, or is the model a smaller one that can run on a personal computer?
