Deploying generative artificial intelligence (GenAI) models or large language models (LLMs), such as ChatGPT-4 and others, in a corporate or other organizational environment presents a unique set of challenges, both technical and human-centric. This pair of blogs looks into the different considerations that both business and security teams must address to ensure they create a safe, secure ecosystem across the enterprise.
First, we look at the technical considerations.
Data Security and Privacy
Ensure that the model is never sent, or otherwise gains access to, confidential, proprietary, or personally identifiable information (PII) unless absolutely necessary, and even then do so with extreme caution. Once the information leaves the confines of your system, there is no way to retrieve it. Deploying automated search-and-block or search-and-redact applications can go a long way toward ensuring that sensitive information or intellectual property (IP) never leaves your system via a prompt to a public LLM. Encryption and other secure data-handling protocols must also be in place. The deployment environment and method must also be taken into effect, due to their inherent security variability and vulnerabilities.
Model Inference Monitoring
Ensure your organization has the capability to monitor real-time inferences (predictions) made by the model, whether that means developing a solution in-house or bringing in experts to advise you or create a tool for you. Monitoring is crucial for understanding how the model is behaving, what data it is analyzing, and whether it is making any incorrect or inappropriate predictions. Depending on the industry, poor performance could produce results ranging from disappointments to disasters, and no organization wants to experience either, if they can be avoided.
Scalability and Performance
LLMs require significant computational resources. While system capacity is always a consideration when the system is being configured and deployed, underestimating the actual need is not uncommon—and scaling up is not an on-demand action. When deploying such models at scale, it’s crucial to consider how well the model performs under load and whether the infrastructure can scale to manage increased demand. When anticipating the need to scale up to accommodate more users, review the hardware and architecture for possible optimizations. More users means more queries, which could overload the system. This can lead to decreased quality of responses, latency issues, or even failures if the system is not adequately scalable.
Version Control and Updating
It’s essential to have a robust version-control system in place before the model is updated or fine-tuned. Changes must be properly documented and tested before being deployed into the live environment. Failing to fully document changes, which means not just what was changed, but who wrote the new or updated code, who deployed the new code and when, and who approved the change; or how the model, training data, user or performance parameters, or anything else changed and who initiated, executed, and approved the change(s). Testing must also be documented with information about the test parameters, thresholds, and outcomes. While documentation is often considered a necessary evil in the software development lifecycle, it’s critical to have the information available to review if something goes wrong down the line, whether that something causes a minor glitch in the user interface or a brand-damaging error in decision-making.
APIs and Integration Security
If the model is being accessed via an API or is integrated with other systems, ensuring secure API endpoints and data encryption is critical to maintaining confidentiality and compliance with regulations like GDPR or HIPAA. Using methods like API keys, OAuth, or other secure authentication mechanisms can help ensure that only users with valid permission status gain access to the model(s). While some authentication features can add latency, lax security can make the system vulnerable to attacks like data breaches and unauthorized access.
Taking these technical factors into account will allow your company to more effectively and securely deploy LLMs across the enterprise. Our next blog discusses the top five human-centric considerations for a secure LLM deployment, so be sure to look for it!
