Lab 9: Web Deployment (Full Stack Deep Learning - Spring 2021)

Lab 9 turns a trained paragraph text recognizer into something that can be called over HTTP and packaged for deployment. The core move is speeding up inference with TorchScript, then exposing the model through two deployment paths: a local Flask web server running in Docker, and an AWS Lambda-style serverless function running locally in a container. The practical payoff is clear—once the model is scripted and wrapped behind an API, the same prediction logic can serve requests from curl (or a client app) and be shipped to production-like environments.

The lab starts by upgrading the Lab 8 PyTorch model for faster inference using TorchScript. TorchScript converts dynamically defined PyTorch code into a statically compiled form, leveraging optimizations that typically reduce inference latency. The changes are intentionally small: the model is set to eval mode, scripted via torch.jit.script, and then inference calls use the scripted model exactly like the original. The lab notes that scripting takes a few seconds up front, but repeated inference becomes faster—especially valuable when a service will handle many requests.

Next comes the web wrapper. A Flask app initializes the model at startup, sets up logging, and defines routes. A simple root route returns “hello world,” while the key endpoint is /v1/predict. That endpoint supports both GET and POST. For POST requests, the server expects a JSON body containing a base64-encoded image; it decodes the image, runs prediction, computes summary statistics, logs the request, and returns the prediction as a string. For GET requests, the server reads an image URL from a query parameter (after the ? in the request URL), fetches the image, and runs the same prediction flow. Supporting both patterns mirrors common real-world API usage.

To keep the service testable, the lab adds a test that sends a request and asserts the response matches expectations, runnable with pytest (via “python -m pytest” / “pytest” style). It emphasizes that web servers aren’t special code-wise—standard unit/integration testing still applies.

Deployment readiness then shifts to packaging. The lab builds a Docker image that includes only production dependencies. The Dockerfile uses a Python 3.6 base image, installs requirements from a production-only requirements.txt, swaps GPU-oriented PyTorch packages for CPU versions, copies the text recognizer and API server code, exposes port 8000, and launches the Flask app. Docker layer caching is highlighted: placing dependency installation steps before copying frequently changing code speeds up rebuilds.

Finally, the lab prepares a serverless version using AWS Lambda conventions. A separate app.pi defines a handler that loads the model, reads an image URL from the incoming event, performs prediction, and returns the result. A Lambda-compatible Docker image runs the handler locally (using the canonical localhost:9000 invocation path). The lab reports timing and billing behavior consistent with Lambda’s execution model, and notes that this approach can integrate with S3 triggers, add a lightweight API gateway in front, and support monitoring in a later lab.

In short: TorchScript reduces inference cost, Flask provides an HTTP interface with GET/POST image inputs, Docker makes the environment reproducible, and the Lambda handler pattern sets up serverless deployment with minimal additional code.