Software development with AI is becoming more and more popular. AI can help developers write code faster, debug issues quicker, and improve overall productivity. However, in an environment where privacy and security are deep concerns such as UVic, we need to be more careful about how AI is used.
Our team recently investigated using AI for log anomaly detection. One of the several approaches was querying Large Language Models (LLMs) through the Ollama API. This article documents the journey from initial exploration to a production-ready API call, including the tools we discovered, the prompt engineering techniques that worked, and best practices for the team moving forward.