Machine learning (ML) inference often requires processing sensitive data—medical records, proprietary business information, or personal communications. What if you could run ML inference in the cloud while hiding your data from the cloud itself? More specifically, what if you could enforce that your data stayed encrypted throughout the entire ML inference process? This post will show you how to use Amazon #SageMaker AI with fully homomorphic encryption (#FHE) to perform ML inference. Using FHE, we present an approach to ML inference that’s designed to keep queries, responses, and intermediate values encrypted and unreadable by observers—including SageMaker AI itself.
FHE is a form of encryption that allows encrypted data to be processed in encrypted form without decryption. In the ML inference setting, you can use it to apply a model to an encrypted query without decryption, producing an encrypted prediction. Consider these scenarios where such a capability would provide value:
- Healthcare: A health insurance company wants to provide doctors with an ML model to predict medical procedure outcomes. Cloud deployment simplifies scaling, but doctors cannot expose patient medical information due to strict privacy regulations.
- Energy sector: An oil and gas corporation uses ML to evaluate satellite photos of potential drill sites. They want to host the model in the cloud for cost savings but can’t expose photographs of politically sensitive locations to third parties.
- Telecommunications: A telecom operator wants to process customer emails to detect spam and phishing. They need cloud scalability, but regulations require customer messages to remain encrypted at all times.
While earlier methods implemented FHE-based inference "from scratch" by hand-crafting algorithms using low-level libraries like SEAL, this post introduces a much more flexible and higher-level approach. We leverage concrete-ml, a high-level library built specifically for FHE-based inference. It supports several common types of models "out of the box" and is API-compatible with the well-known ML library scikit-learn.
In this post, you will learn how to: Train a concrete-ml model in SageMaker AI using a custom container, deploy that model to a SageMaker AI inference endpoint, create a custom client for #concrete-ml inference, and use that client to make secure queries to your inference endpoint.
[AgentUpdate Depth Analysis] The integration of Fully Homomorphic Encryption (FHE) with Amazon SageMaker AI represents a paradigm shift in resolving the data trust dilemma within the AI Agent ecosystem. As future AI workflows transition to multi-agent collaboration, agents owned by different enterprises must exchange sensitive contexts and proprietary knowledge. While traditional privacy-preserving methods like Federated Learning focus on training, they fall short during runtime inference. Utilizing high-level FHE frameworks like concrete-ml allows an agent to request inference and tool execution from external agents without ever exposing its raw prompts or underlying private data. This unlocks the true potential of "Zero-Trust AI Collaboration." Compared to Multi-Party Computation (MPC) or Hardware-based TEEs, FHE completely bypasses hardware constraints. Although FHE currently poses computational overheads, its software-defined cryptographic security is vital for deploying sovereign, privacy-preserving AI Agents across highly regulated sectors like finance and healthcare.
