Deploying Hugging Face Models on AWS SageMaker

In this blog, we delve deep into the flexible and economical methods available for deploying Hugging Face models on AWS SageMaker. Whether you are operating through the SageMaker Notebook/Studio or your personal laptop, and independent of your internet accessibility status, this guide helps you navigate the most effective paths to deploy your models.

Hugging Face remains the most substantial repository, harboring a wide variety of deep learning model collections, datasets, and libraries. It also hosts benchmarks for these models, an essential feature in the dynamically evolving space of LLMs. Some popular models hosted at this point include Llama-2 (Meta), Falcon-180 (tiiuae), stable diffusion (stabilityai), and many more, including sentence-transformers which are a great tool for word embeddings.

The AWS SageMaker SDK stands as a powerful tool in your arsenal when working with Hugging Face, aiding in various processes such as:

• Training Models: Initiate the process of building models suited for your specific datasets.
• Fine-Tuning Models: Adapt pre-trained Hugging Face models to meet distinct requirements, enhancing their utility and adaptability to various datasets.
• Deployment for Inferencing: Leverage the SageMaker SDK with Hugging Face to directly specify the Hugging Face token, model name, and essential parameters, simplifying the deployment process on AWS.

Simplicity: The SDK offers a straightforward route to deploying models, reducing complexity.

Security: AWS ensures a secure environment, safeguarding your models throughout the deployment.

Utilizing SageMaker Studio/Notebook undoubtedly facilitates script creation by handling numerous nuances such as IAM role, region, and SageMaker session configurations. However, this comes at a continuous running cost during the development, deployment, and management of scripts.

A cost-efficient alternative involves leveraging the same SDK from a notebook operated from your laptop, substantially reducing deployment expenses. Other viable options include utilizing an EC2 instance or Cloud9, which, while incurring some costs, remain significantly cheaper than operating through SageMaker Studio/Notebook.

Depending on your accessibility to the internet, you have two options to choose from for deploying your models:

• Option A: If you have direct access to the internet, leverage the Hugging Face HF_MODEL_ID to deploy the model using the SageMaker SDK integrated with Hugging Face. Detailed guidance on this can be found here.
• Option B: In scenarios where internet access is unavailable, opt to deploy the model within a VPC by first downloading it to S3. From there, load the model into SageMaker utilizing the Hugging Face LLM Inference container. For a comprehensive walkthrough, you may refer to this guide.

In this blog, we outlined practical methods to deploy Hugging Face models on AWS SageMaker, a process possible both through the SageMaker environment and a local laptop. The guide also caters to situations with varying levels of internet access, ensuring you have the knowledge to proceed under different circumstances. We hope this aids you in utilizing Hugging Face models more efficiently and economically.

For further reading and understanding, you may refer to the following resource:

• Phil Schmid’s Blog

Written By: Umashankar N

If you have any questions or suggestions, please reach out to us at contactus@1cloudhub.com