Sunday Coffee & Code - Claude Code Terminal and Local Gemma4 (yup, more Gemma)

𝗦𝘂𝗻𝗱𝗮𝘆 𝗖𝗼𝗳𝗳𝗲𝗲 & 𝗖𝗼𝗱𝗲 - 𝗖𝗹𝗮𝘂𝗱𝗲 𝗖𝗼𝗱𝗲 𝗧𝗲𝗿𝗺𝗶𝗻𝗮𝗹 𝗮𝗻𝗱 𝗟𝗼𝗰𝗮𝗹 𝗚𝗲𝗺𝗺𝗮𝟰 (𝘆𝘂𝗽, 𝗺𝗼𝗿𝗲 𝗚𝗲𝗺𝗺𝗮)

Not a lot of time this weekend so grab your coffee - this week mixed ”𝘤𝘢𝘯 𝘊𝘭𝘢𝘶𝘥𝘦 𝘊𝘰𝘥𝘦 𝘥𝘳𝘪𝘷𝘦 𝘢 𝘭𝘰𝘤𝘢𝘭 𝘮𝘰𝘥𝘦𝘭?” (which, tbh, we know it could before starting) with ”𝘸𝘢𝘵𝘤𝘩 𝘢 31𝘉 𝘮𝘰𝘥𝘦𝘭 𝘨𝘳𝘪𝘯𝘥 𝘵𝘰 𝘢 𝘩𝘢𝘭𝘵 𝘰𝘷𝘦𝘳 𝘢 𝘤𝘰𝘯𝘵𝘦𝘹𝘵 𝘸𝘪𝘯𝘥𝘰𝘸.” (… and a confession: I had tea today. Don’t tell the branding.)

I’ve been using Claude Code across Web, Windows, and Terminal modes (mostly Web), and after some airport lounge testing with a Gemma model on my laptop, I wanted to put the two together - Claude Code on my AWS server (g6.2xlarge), pointed at local Gemma4 models via Ollama.

The test case: a Code Audit prompt from GitHub (link at the bottom of the post - Roger Faught - is this your prompt? ), run against some old C code of mine - the same Unix program I used for earlier Gemma testing.

𝗙𝗶𝗿𝘀𝘁, 𝘁𝗵𝗲 𝗯𝗮𝘀𝗲𝗹𝗶𝗻𝗲:

• Claude Code + GitHub CLI + Sonnet: it just ran. Full audit in about 6 minutes, consuming roughly 3% of my Pro subscription. Straightforward.

𝗧𝗵𝗲𝗻 𝘁𝗵𝗲 𝗳𝘂𝗻 𝘀𝘁𝗮𝗿𝘁𝗲𝗱 𝘄𝗶𝘁𝗵 𝗹𝗼𝗰𝗮𝗹 𝗺𝗼𝗱𝗲𝗹𝘀:

• Gemma4:31b loaded onto the GPU fine (confirmed via ollama ps), but then the usual num_ctx problem showed up. Bumping the context window to 128k - and 64k - split the model ~50/50 between CPU and GPU, and everything ground to a halt.
• Dropped to gemma4:12b, pushed the context window back up to 128k, and it stayed fully on the GPU. Interesting detail: going from 64k to 128k num_ctx bumped memory consumption from 8.4GB to 9.5GB. The audit ran in about 10 minutes and produced risk-based recommendations on the codebase - success. (𝘐𝘵 𝘸𝘰𝘶𝘭𝘥 𝘣𝘦 𝘪𝘯𝘵𝘦𝘳𝘦𝘴𝘵𝘪𝘯𝘨 𝘵𝘰 𝘵𝘳𝘺 𝘵𝘩𝘦 26𝘉 𝘔𝘰𝘌 𝘮𝘰𝘥𝘦𝘭)

𝗪𝗵𝗮𝘁 𝗜 𝗹𝗲𝗮𝗿𝗻𝗲𝗱:

• Ollama + Claude Code Terminal + GitHub setup is straightforward. Once the context window is sorted - that’s been the recurring lesson with local models: num_ctx is almost always the first thing to check.
• There may be a real fit for local models in my workflow: use them for smaller, simpler scripts, and save the larger frontier models (and my subscription budget) for the work that actually needs them.
• Purpose achieved. The script ran, the audit completed, and I’ve got a working local option.

Curious how you’re drawing the line between subscription and local models with tools like Claude Code (Qwen Coder anyone?)

𝗔 𝗴𝗼𝗼𝗱 𝗦𝘂𝗻𝗱𝗮𝘆 𝗮𝗻𝗱 𝗺𝘆 𝘁𝗲𝗮’𝘀 𝘀𝘁𝗶𝗹𝗹 𝘄𝗮𝗿𝗺.

• Code Audit prompt: GitHub Gist Link
• The C code under audit: GitHub - Menu Program