Most smart homes are basically very expensive light switches with a voice - โ๐๐ถ๐ณ๐ฏ ๐ฐ๐ฏ ๐ต๐ฉ๐ฆ ๐ฑ๐ฐ๐ณ๐ค๐ฉ ๐ญ๐ช๐จ๐ฉ๐ต.โ
This Agent and physical world integration experiment continues, connected to Spotify and lightingโฆ then, tell it how I felt.
So I added mood mode. A little text face made entirely of text dots appears in my terminal (ASCII ftw), blinks, raises an eyebrow while itโs thinking, and asks out loud:
๐จ๏ธ โ๐๐ช! ๐๐ฐ๐ธ ๐ข๐ณ๐ฆ ๐บ๐ฐ๐ถ ๐ง๐ฆ๐ฆ๐ญ๐ช๐ฏ๐จ ๐ณ๐ช๐จ๐ฉ๐ต ๐ฏ๐ฐ๐ธ?โ
Me: โ๐โ๐ฎ ๐ง๐ฆ๐ฆ๐ญ๐ช๐ฏ๐จ ๐ฑ๐ณ๐ฆ๐ต๐ต๐บ ๐ต๐ช๐ณ๐ฆ๐ฅ.โ
๐ค The bulb fades to a soft amber, a calm acoustic playlist starts, and it says: โ๐๐ฐ๐ถ๐ฏ๐ฅ๐ด ๐ญ๐ช๐ฌ๐ฆ ๐ข ๐ญ๐ฐ๐ฏ๐จ ๐ฅ๐ข๐บ. ๐โ๐ท๐ฆ ๐ด๐ฆ๐ต ๐ข ๐ด๐ฐ๐ง๐ต ๐ข๐ฎ๐ฃ๐ฆ๐ณ ๐จ๐ญ๐ฐ๐ธ ๐ข๐ฏ๐ฅ ๐ฑ๐ถ๐ต ๐ฐ๐ฏ ๐ต๐ฉ๐ฆ ๐ฅ๐ณ๐ข๐ช๐ฏ๐ฆ๐ฅ. ๐ฑ๐ญ๐ข๐บ๐ญ๐ช๐ด๐ต, ๐ด๐ต๐ข๐ณ๐ต๐ช๐ฏ๐จ ๐ธ๐ช๐ต๐ฉ ๐๐ญ๐ต๐ณ๐ข๐ท๐ช๐ฐ๐ญ๐ฆ๐ฏ๐ค๐ฆ ๐ฃ๐บ ๐๐ข๐ฏ๐ข ๐๐ฆ๐ญ ๐๐ฆ๐บ.โ
Me: โ๐๐ฆ๐ฆ๐ญ ๐ญ๐ช๐ฌ๐ฆ ๐ฎ๐ฐ๐ณ๐ฆ ๐ญ๐ช๐ฌ๐ฆ ๐ข ๐ฑ๐ข๐ณ๐ต๐บ ๐ข๐ต๐ฎ๐ฐ๐ด๐ฑ๐ฉ๐ฆ๐ณ๐ฆ ๐ฏ๐ฐ๐ธ. ๐๐ฆ๐ตโ๐ด ๐จ๐ฆ๐ต ๐ด๐ฐ๐ฎ๐ฆ ๐ฑ๐ข๐ณ๐ต๐บ ๐ฎ๐ถ๐ด๐ช๐ค ๐จ๐ฐ๐ช๐ฏ๐จโ
๐ค โ๐๐ฌ๐ข๐บ, ๐โ๐ท๐ฆ ๐ค๐ฉ๐ข๐ฏ๐จ๐ฆ๐ฅ ๐ต๐ฉ๐ฆ ๐ญ๐ช๐จ๐ฉ๐ต๐ช๐ฏ๐จ ๐ต๐ฐ ๐ข ๐ฃ๐ณ๐ช๐จ๐ฉ๐ต๐ฆ๐ณ ๐ฎ๐ข๐จ๐ฆ๐ฏ๐ต๐ข ๐ข๐ฏ๐ฅ ๐ฑ๐ถ๐ต ๐ฐ๐ฏ ๐ต๐ฉ๐ฆ 100 ๐ฐ๐ง ๐๐ฉ๐ฆ ๐๐ฆ๐ด๐ต ๐๐ข๐ณ๐ต๐บ ๐๐ฐ๐ฏ๐จ๐ด ๐ฑ๐ญ๐ข๐บ๐ญ๐ช๐ด๐ต, ๐ด๐ต๐ข๐ณ๐ต๐ช๐ฏ๐จ ๐ธ๐ช๐ต๐ฉ ๐๐ช๐ด๐ค๐ฐ ๐๐ฏ๐ง๐ฆ๐ณ๐ฏ๐ฐ - ๐๐ฅ๐ช๐ต ๐ฃ๐บ ๐๐ฉ๐ฆ ๐๐ณ๐ข๐ฎ๐ฎ๐ฑ๐ด.โ
I never said โ๐ต๐ถ๐ณ๐ฏ ๐ฐ๐ฏ ๐ต๐ฉ๐ฆ ๐ญ๐ช๐จ๐ฉ๐ตโ. I never named a playlist. It worked out the vibe on its own. That was the whole point of the experiment: moving agents from following instructions to interpreting what you need, ๐๐ถ๐๐ต ๐ฟ๐ฒ๐ฎ๐น ๐ฑ๐ฒ๐๐ถ๐ฐ๐ฒ๐ in the loop .
Lessons from the lab ๐งช
- MCP is magic glue. Once the bulb was an MCP server, adding music was just plugging in a second server, with no new integration code.
- Personality was interesting, the ascii face added a little personalisation.
- Kept it local where I could. Speech recognition and the voice run on my own PC. Only the thinking, the weather and Spotify go out to the internet.
The kit list ๐ ๏ธ
- Microsoft Agent Framework: the agentโs brain wiring
- Model Context Protocol (MCP): a custom WiZ bulb server, plus a community Spotify MCP server
- Gemini 2.5 Flash via OpenRouter: the reasoning (Ollama for local models)
- faster-whisper: speech-to-text, on my PC
- Piper: a natural-sounding voice, also on my PC
- Spotify Web API: the soundtrack (Premium needed)
- WiZ smart bulb: the star of the show
- OwnTracks + Mosquitto (MQTT) + Tailscale: bonus feature, the porch light turns on as I get close to home
- OpenWeather: so it knows when itโs dark and whether itโs raining
- Python on WSL2 (Windows 11), pair-programmed with Claude Code
Perhaps the future of home agents isnโt more commands, itโs fewer. Thatโs this run of agent and physical world integration experiments over with - on to something new next week. Time to grab another coffee - remember: Think-Model-Act. โ