Our team recently investigated using AI for log anomaly detection. One of the several approaches was querying Large Language Models (LLMs) through the Ollama API. This article documents the journey from initial exploration to a production-ready API call, including the tools we discovered, the prompt engineering techniques that worked, and best practices for the team moving forward.