I've been feeling melancholy today, for a number of reasons, so I needed something to kind of dig into that's quasi mindless but at least in some way productive. I saw a story about a particular open source vulnerability today, so it seemed like a good time to update all of the things. It's a boring task, but it requires a lot of testing in case the change or fix breaks something else. Most of the time it doesn't, but if what you have is too far out of date, it'll likely break. AI isn't great for this, because it just doesn't have enough context in the moment to know what to look for.
On the other hand, I had some free extra credits from Claude (they sure know how to bleed cash), so I unleashed it on TogetherLoop to find security problems. It found four, but only one was serious, and not one that I would have ever thought about up front. But the funny thing is, it was in code that it had written. This is one of the many strange things about agentic coding, that it does a lot of dumb things, unless you ask it to check for the dumb things.
I had it scan POP Forums, too, and it found some minor things around possible script injection attacks. The app as a whole, having evolved over so many years, has a lot of competing actions. Parsing text is always kind of weird, and it's the worst part of the code. But also, sometimes it parses stuff then stores it, other times it stores it then parses it. That's where the risk was. Meh, no one else saw it, so I guess it's fine-ish. All fixed now.
These AI agents are fantastic, but the LLM's that they use are definitely at risk to rot. They're trained on all kinds of information and code all over the Internet, without the wisdom to know what's good and what isn't. But also, if those sources disappear, and they probably will if no one uses them, what does it train on? If it trains on itself, it will devolve and suck. Google's AI summaries present the same problem. If all of its source material disappears, because no one actually follows the links to it, then what?
No comments yet.