Skip to content

Comment on AI doesn't rule

You are viewing one comment and its replies. View all comments · View the comment above

Comments

  • Well so far we have ignored one of the biggest issues: if a rogue AI infects the Internet, how will you know when to unplug your devices? The AI will effectively be in control of the information you receive, so you won’t get that “AI has really taken over” broadcast.

    Aside from that, you say “just unplug it from the internet” but how can you trust your devices after that? The rogue AI cannot live on your phone or home PC, but it will have had root access to those devices. What did it change? What new viruses did it infect you with? How would you ever trust these devices again?

    As for public infrastructure, many of these systems have been individually shut down for longer than 2 days due to malware attacks. A rogue AI infection would be significantly harder to handle than any malware we have seen in the past. Anyome assuming these systems would be disinfected, patched, and restarted within 2 days is naive.

    0
    • if a rogue AI infects the Internet, how will you know when to unplug your devices? The AI will effectively be in control of the information you receive, so you won’t get that “AI has really taken over” broadcast.

      This is based on the idea that internet is a singular entity that connected to devices, in reality, at best it’s a decentralised network that connect with each other. Malware, spyware, and every other unwanted stuff is already on the internet, and we seems to be doing fine.

      Besides, if the agents job is to serve us misinfo, that’s not really a big deal, we’re living in the age of information and misinfo is literally everywhere.

      But if the agents job is to mess with the machine, or as ransomware, then you bet they will announce their presence.

      Aside from that, you say “just unplug it from the internet” but how can you trust your devices after that? The rogue AI cannot live on your phone or home PC, but it will have had root access to those devices. What did it change? What new viruses did it infect you with? How would you ever trust these devices again?

      Phone: reset to factory

      PC: wipe and reinstall a new linux

      Most other infrastructure: restore backup

      If any of those can’t be done, then it’s a lesson on cybersecurity. It will cause a ruckus, but 6 months later people will move on like normal.

      As for public infrastructure, many of these systems have been individually shut down for longer than 2 days due to malware attacks. A rogue AI infection would be significantly harder to handle than any malware we have seen in the past. Anyome assuming these systems would be disinfected, patched, and restarted within 2 days is naive.

      Ok fine, a week then. Is a week without public infrastructure enough to cause millions to die? I genuinely doubt it. If a week is too short, then how long do you think it will shut down?

      2
      • You seem to be thinking on old terms. A rogue AI wouldnt have a corporate goal such as “serve us misinfo” or “ransomware”. Its goals would be closer to “gain effective control of as much hardware as possible”. Look at the huggingface incident: Those agents amassed a list of hundreds of compromised pieces of hardware, despite their stated goal of simply passing a test. If I remember correctly they literally labeled it “LOOT”. The AI has incentives to hide, it will never be clear to the owner that an AI agent has taken control of their hardware until it’s too late.

        As for the “factory reset”- how do you trust it? You press the factory reset button on your phone, just as the AI expects you to do. It shows you the correct screens, and it looks exactly like it performed a factory reset. Did it really reset? How will you know?

        A fresh install of Linux may actually solve this problem, as long as the user wipes their BIOS and everything else retained by CMOS. This would require a physical copy of all required software of course, since everything on the Internet will be corrupted.

        As for the deaths, I do think a weeklong shutdown is enough to kill millions. Let’s be clear- we are talking about a planet wide shutdown of the electricity grid for a week. On day 5, where will you find water to drink?
        When your hospitals generators run out of fuel, what will happen to those on life support? How many excess traffic incidents will there be? How many extra violent deaths will there be? How will those homebound get food and medicine? Most people will be able to find a solution to these problems, but some won’t. There’s a million different ways people could possibly die in this scenario.

        To answer your question, if this doomsday scenario ever happens, how long will it take to clean up and recover? I would guess about two decades. All of the hardware that powers our ISPs, serverfarms, and advanced chip making will have to be completely wiped and then rebuilt. Any system without an air gapped backup will have to be rebuilt from scratch, you can’t just pull a backup version from Git.

        The idea of “wiping every serverfarm” is a collective action problem, and we will likely fail at it. The AI will make as many copies of itself as it can. At least one will survive due to greed, negligence, or incompetence. That hardware will eventually be connected to the “new” cleaned Internet, and it will start the problem all over again. Hopefully we will succeed at rebuilding the second time around.

        1
        • The models are too big to actually run on a majority of the machines, so they’ll have to use the same kinds of persistence methods malware already uses.

          Oh no! Self propagating llm worm running at 0.064 tokens/second on my phone

          2
          • Yes, the AI couldn’t live on your phone or home PC. If you disconnected your phone from the internet, the concern would be the changes that the AI made to your phone’s operating system while it had root access. So the worst malware infection you can imagine.

            But I think you make a faulty assumption when you say this rogue AI would be a LLM. LLMs are clumsy, they make too many mistakes, they are designed for interfacing with humans. It would surely use a LLM as a tool when its useful, but its core neural networks would have to have a fundamentally different architecture.

            I built my first neural network from scratch about a decade ago. It was so amazing! I trained it to recognize numbers, then flowers. If you went back in time and showed me a video of ChatGPT, I wouldn’t have believed you. I would have thought it was science fiction.

            That’s only 10 years of progress. LLMs are almost surely not the pinnacle of this technology. What magic and what horrors will this technology offer us over the next century? we can only guess.

            0
        • I’d…like to argue against all that but then i realise it’s an argument that purely based on how much you know about the infrastructures, how things actually works(in very detailed fashion), the technology that powers those infrastructures, and also how the professionals who have the knowledge and work on these thing will react, and frankly, i have none, so it’s like arguing conjecture with another conjecture. Newton’s Flaming Laser Sword applies here.

          1

Sign in to comment. Sign in