Effectively managing the #context window is a critical challenge when utilizing large AI models for log diagnosis. The HiyokoLogcat project recently shared its best practices for sending log lines to the #Gemini model for troubleshooting, based on practical tests. The goal was to balance context richness with token limits, response speed, and signal-to-noise ratio.
In initial attempts, the project team simply grabbed the last 500 log lines and sent them to Gemini 1.5 Flash. Despite Gemini's large context window, 500 verbose #logcat lines generated a significant number of tokens, leading to noticeably increased response latency. Furthermore, the model sometimes focused on unrelated noise rather than the actual error.
After extensive testing, HiyokoLogcat discovered a "sweet spot": sending ±50 lines around the target error, totaling approximately 100 lines. This approach effectively:
- Captures the sequence of events leading to the crash.
- Records any follow-on errors and recovery attempts.
This window size typically results in 3,000 to 6,000 tokens, yielding fast responses and relevant diagnostic results. Programmatically, a function (e.g., `get_diagnosis_context`) can be used to extract context lines around an error index and format the logs into a more compact string.
However, certain complex scenarios, such as memory leaks that build up over minutes or threading issues with long setup chains, necessitate a wider context. For these cases, HiyokoLogcat introduced a "deep diagnosis" mode, which sends ±200 lines around the error. The quick diagnosis (±50 lines) remains the default, with deep diagnosis offered as an optional feature for specific needs.
Beyond context window size, log format is also crucial. Raw logcat output often contains a lot of repetitive structure. Reformatting it before sending to the model can significantly reduce token count without losing any critical information. For instance, condensing verbose raw log entries (e.g., `04-20 10:23:45.123 1234 5678 E AndroidRuntime: FATAL EXCEPTION: main`) into a more compact format (e.g., `[10:23:45] E/AndroidRuntime: FATAL EXCEPTION: main`) proves effective in saving tokens when dealing with large volumes of logs.
Ultimately, by combining ±50 lines of context, reformatted logs, and PII masking, HiyokoLogcat enabled Gemini to provide specific, actionable diagnoses within 2-3 seconds on the free tier. This configuration has been integrated and shipped with the HiyokoLogcat project.
