Effectively managing the context window is a critical challenge when utilizing large AI models for log diagnosis. The HiyokoLogcat project recently shared its best practices for sending log lines to the Gemini model for troubleshooting, based on practical tests. The goal was to balance context richness with token limits, response speed, and signal-to-noise ratio.
In initial attempts, the project team simply grabbed the last 500 log lines and sent them to Gemini 1.5 Flash. Despite Gemini's large context window, 500 verbose logcat lines generated a significant number of tokens, leading to noticeably increased response latency. Furthermore, the model sometimes focused on unrelated noise rather than the actual error.
After extensive testing, HiyokoLogcat discovered a "sweet spot": sending ±50 lines around the target error, totaling approximately 100 lines. This approach effectively:
- Captures the sequence of events leading to the crash.
- Records any follow-on errors and recovery attempts.
This window size typically results in 3,000 to 6,000 tokens, yielding fast responses and relevant diagnostic results. Programmatically, a function (e.g., `get_diagnosis_context`) can be used to extract context lines around an error index and format the logs into a more compact string.
However, certain complex scenarios, such as memory leaks that build up over minutes or threading issues with long setup chains, necessitate a wider context. For these cases, HiyokoLogcat introduced a "deep diagnosis" mode, which sends ±200 lines around the error. The quick diagnosis (±50 lines) remains the default, with deep diagnosis offered as an optional feature for specific needs.
Beyond context window size, log format is also crucial. Raw logcat output often contains a lot of repetitive structure. Reformatting it before sending to the model can significantly reduce token count without losing any critical information. For instance, condensing verbose raw log entries (e.g., `04-20 10:23:45.123 1234 5678 E AndroidRuntime: FATAL EXCEPTION: main`) into a more compact format (e.g., `[10:23:45] E/AndroidRuntime: FATAL EXCEPTION: main`) proves effective in saving tokens when dealing with large volumes of logs.
Ultimately, by combining ±50 lines of context, reformatted logs, and PII masking, HiyokoLogcat enabled Gemini to provide specific, actionable diagnoses within 2-3 seconds on the free tier. This configuration has been integrated and shipped with the HiyokoLogcat project.