Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
Design Guide—Generative AI in the Enterprise - Inferencing
Introduction AI Model Inferencing Architecture Concepts and Components Solution Architecture Validation Results Performance Characterization and Sizing Inferencing Operations
Model deployment
Life cycle
Backup and restore
Secure model serving
Model monitoring
Dell Professional Services for Generative AI Conclusion References

Secure model serving

Secure model serving

  • Securing AI models in production is crucial to protect sensitive data, maintain system integrity, and prevent unauthorized access. Important security considerations for AI models in production include:

    • Secure model serving—When deploying the AI model, employ secure containerization and sandboxing techniques to isolate the model from the underlying infrastructure and prevent potential attacks. Docker and Kubernetes provide these mechanisms.
    • Authentication and authorization—Implement strong authentication mechanisms to verify the identity of users and applications accessing the AI model. Use access control lists (ACLs) or role-based access control (RBAC) to enforce appropriate permissions for different user roles. Kubernetes deployed through NVIDIA’s cluster manager supports both ACL and RBAC to provide fine-grained authorization and control over GPU resources and AI models in the cluster. These mechanisms help enforce security policies and ensure that only authorized users and processes have access to perform inference operations on Kubernetes resources.
    • Secure APIs—Triton Inference Server exposes the AI models through APIs. These APIs can be secured using authentication and authorization mechanisms. Use tokens or API keys to validate requests and prevent unauthorized access.
    • Secure Communications—Triton Inference Server supports secure communication protocols, such as HTTPS, to protect data transmission between clients and the AI model under inference.