How worried should I be about running LLM code on my machine?
2 points by scoofy 15 hours ago | 4 comments
Look, I'll be honest about being an AI skeptic generally, but I recently thought I'd give LLM assisted development a try and it's been wildly successful for me. On the project I've been working on, it feels like I've gotten multiple months worth of work done in a week. Most of that time would have been spent researching some statistical methods for doing things, and it's much quicker to verify a method as being accurate, than it is to find one that works and learn how to create an implementation.

However, this has still had me running code in my computer that is, to say the least, dubious. I'm using Google Gemini Pro, and while I keep asking the LLM to only give me chunks of code that it's changing, it keeps asking me to just replace, say, the main.py file completely. This has me genuinely concern about running arbitrary code on my machine, intentionally or just due to laziness.

My question here is (1) how much of a risk am I taking on running this kind of code when it's not at all related to my internal filesystem, (2) what are mitigation strategies I can take to protect myself beyond regular backups short of just doing everything in, say multipass or UTM, and (3) if I need to run a virtual machine for everything, which is the easiest to run for moving back and forth between normal use on a mac?

Really appreciate any advice here. I'm mostly a hobbyist programmer, even if I've been developing stuff in Python for a decade and a half a this point.

  • dpforesi 14 hours ago |
    I am a professional LLM code tester. I cannot say much more due to my NDA, but if you are very concerned I would run it in a docker container at the very least. That being said, I highly suggest reviewing the code carefully before running it. Even a look at say python imports might tell you what the code can potentially do if you see it importing the os module. I've run advanced code agents in vs code using their unrestricted access settings while in a docker container. At first it was scary, then I started using that time to refill my coffee. My computer is still running fine.
    • scoofy 14 hours ago |
      Looking at the imports is very good advice, thanks.
  • forgotpwd16 14 hours ago |
    There's a risk but imo think there's some overreaction. Plus should keep backups anyway. Have had few disks failing on me that fear to have any important data with single copy. If want to be on the safe side though: Create container/VM. Do agentic work inside. Git commit/push outside often/every-step.
  • forgotpwd16 an hour ago |