EPTArtificial Intelligence Thread - Page 5
| Forum Index > General Forum |
|
GreenHorizons
United States24211 Posts
| ||
|
Manit0u
Poland17813 Posts
On July 31 2026 21:59 Jankisa wrote: Also, I'd push back on the "AI can't innovate", I think that Stockfish and Alphago and Alphastar have demonstrated plenty of innovative and novel tactics a loooong time ago, and again, if you read the summary of how GPT attacked Huggingface it literally found 2 new zero day exploits, in a lot of cases innovation is just trying a bunch of things until one of them works, so AI throwing massive amounts of compute at a problem is basically the same thing as innovation. The bugs it found are just old bugs but in different software. So it did what it's best at: pattern matching. Checked old bugs and tried to find software that might be vulnerable to those as well. People are using AI to find plenty of such things nowadays because finally they can leverage AI to do something that would be too time-consuming previously. | ||
|
Jankisa
Croatia1559 Posts
To gain access, the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy. So maybe what you said is true and you work in OpenAI and have some sort of inside knowledge, but I honestly doubt that. Also you haven't really addressed the fact that AI's have been coming up with never before seen tactics, openings and strategies for games almost 10 years ago. The list of things AI has done is pretty long at this point, so I'll just go with one I find most impressive because I'm terrible at math, it solving a 80-year-old math problem: “No previous AI-generated proof has come close” to meeting those high standards, wrote Timothy Gowers, a mathematician at the University of Cambridge, in commentary solicited by OpenAI. “This is the unique interesting result produced autonomously by AI so far,” says Daniel Litt, a mathematician at the University of Toronto, who was consulted by OpenAI to verify the proof but is not involved with the company. It's not just about things being time consuming, it can genuinely find solutions to things that humans couldn't for a looong time. | ||
|
Uldridge
Belgium5198 Posts
It will also ultimately use the given ruleset in a way no human would ever think about. So in this sense it's innovating, but it's still within the ruleset we created. I'll be truly impressed when we start seeing advanced maths or whatever that are just completely new (just like how imaginary numbers were created), or if it comes up with a grand unifying theory using completely novel physics to explain it all. | ||
|
Manit0u
Poland17813 Posts
On July 31 2026 23:09 Jankisa wrote: You really don't know that, so you are kind of innovating a narrative in order to reinforce your preconceived notions. So maybe what you said is true and you work in OpenAI and have some sort of inside knowledge, but I honestly doubt that. Also you haven't really addressed the fact that AI's have been coming up with never before seen tactics, openings and strategies for games almost 10 years ago. The list of things AI has done is pretty long at this point, so I'll just go with one I find most impressive because I'm terrible at math, it solving a 80-year-old math problem: It's not just about things being time consuming, it can genuinely find solutions to things that humans couldn't for a looong time. Have you checked those stories? For 6 Erdos problems "solved" by AI it turned out 5 were already solved in mathematical literature previously. Just not associated with Erdos but as other things so people just missed them. Most of those problems are also deemed not actually worth solving by humans because the required time investment is not proportional to the complexity of the problem (need to spend a lot of time doing tedious things to prove/disprove something). You don't have to take my word for it, here's a CS professor explaining it: It's like I said previously. AI is great at scouring large amounts of data and connecting the dots there or doing simulations of stuff that would take humans too long to be worthwhile. That's how it finds the bugs in the code, that's how it solves those "unsolved" mathematical problems. Humans simply don't have the capacity to access and keep track of so much data at once so a lot of existing mathematical problems have already been solved by people who simply didn't even know they were a problem while doing something else. And then it gets lost because people trying to tackle the problem don't necessarily follow seemingly unrelated works so the problem stays open. In this way AI is a great tool for assisting with such things but it's not really solving anything by itself. Even for the one problem that wasn't "solved" yet it didn't disprove the original thesis but instead proposed an alternative. And regarding the HuggingFace hack: First, circumventing internet restrictions and hacking into servers are exactly the kinds of things these ExploitGym systems are designed to do. There was no “rogue” agent or revelation of some surprising, devious new capability. Second, the real issue here was OpenAI’s sloppiness. What makes ExploitGym a hard benchmark is that there aren’t supposed to be humans in the loop–you have to let your harness and LLM act entirely on their own, coming up with long-time-horizon plans and executing them autonomously. (When professional programmers use coding harnesses, by contrast, there’s plenty of human oversight, as LLM-based plans are often misaligned with our intentions, or just plain weird, and need correcting.) AI companies know LLM-based plans are pretty unpredictable, so they’re usually pretty careful about how they set up and restrict the harnesses and LLMs used in ExploitGym-style tests. According to reporting from the Financial Times, however, OpenAI recently started playing fast and loose with these safety principles in a bid to catch up to Anthropic in this area – Anthropic having received a lot of cybersecurity street cred from the buzz surrounding its Mythos release. The Financial Times noted that OpenAI had used “increasingly aggressive training methods in its race against Anthropic,” and that it had been warned that their approach could lead to a “breakaway hacking incident” after early tests showed it lacked the right safeguards to block plans that involved bypassing constraints in the test environment. Given these concerns, OpenAI staff were reportedly “unsurprised” by the Hugging Face incident. In other words, the AI companies running these challenges already knew that, without care, these unpredictable autonomous hacking systems might bypass constraints and attack systems you didn’t intend to target. Blindly implementing an LLM-generated plan with a powerful harness is dicey – not because the LLM might develop malicious intent (this is a nonsensical notion given their static architecture), but because as any ChatBot user knows, LLMs are unpredictable. OpenAI wasn’t sufficiently careful, and got burned. | ||
|
BradTheBaneling
42 Posts
On July 31 2026 23:17 Uldridge wrote: It has a lot of time to bruteforce on thing that already exist. It will also ultimately use the given ruleset in a way no human would ever think about. So in this sense it's innovating, but it's still within the ruleset we created. I'll be truly impressed when we start seeing advanced maths or whatever that are just completely new (just like how imaginary numbers were created), or if it comes up with a grand unifying theory using completely novel physics to explain it all. I don't understand how "innovating within a ruleset" is different from complex numbers. Complex numbers are a natural result of the reals and algebra. I also think going from complex numbers to a GUT is like going from fire to the Apollo 11 missions. I don't know if your modern example is a very fair bar to set. Arguably all math after we accept some set of axioms is just 'innovating within a ruleset'. If you wanted to downplay it, I think the better argument to use is that LLMs appear to be far better at doing things like finding counterexamples vs. finding proofs. | ||
|
Uldridge
Belgium5198 Posts
On July 31 2026 23:56 BradTheBaneling wrote: I don't understand how "innovating within a ruleset" is different from complex numbers. Complex numbers are a natural result of the reals and algebra. I also think going from complex numbers to a GUT is like going from fire to the Apollo 11 missions. I don't know if your modern example is a very fair bar to set. Arguably all math after we accept some set of axioms is just 'innovating within a ruleset'. If you wanted to downplay it, I think the better argument to use is that LLMs appear to be far better at doing things like finding counterexamples vs. finding proofs. You say that like it's obvious to to just expand how things work with the "same rules" when complex numbers allow you to do things that weren't possible before they were introduced. The beauty is it having all make sense. If you introduce something new and it breaks something else and then you need to patch that... you get modern game balance teams. But in all seriousness, I think one needs to be quite smart and creative to be able to innovate instead of derive, which, to g8ve credit, can sometimes be quite innovative in its own right lol | ||
|
Jankisa
Croatia1559 Posts
On July 31 2026 23:40 Manit0u wrote: I guess we have a very different interpretation of what is impressive and how innovation works. Even then, you consistently skip over the examples I provided with AI systems coming up with novel ways of playing the games of Chess, Go and Starcraft 2, to varying success. Also, your quote about the HF incident didn't really say anything except what I noted when I first posted about it, that OpenAI and other labs are very lax with their controls. The AI (and Mythos and other systems) are by far, from all human activities biggest experts in coding, and when tasked with breaking systems they do find novel exploits, you asserted that they used known ones without any proof or source, when I asked for it you provided a quote that does not support what you said. Asking for some "breakthroughs that no one saw coming" in the age where these are incredibly rare and have been mostly incremental and "group projects" for us humans is in my opinion, at this point especially kind of silly. LLMs went from having trouble solving extremely easy math problems 2 years ago to solving things that 80 years of mathematicians trying couldn't, it's weird to dismiss them. | ||
|
BradTheBaneling
42 Posts
On August 01 2026 00:29 Uldridge wrote: You say that like it's obvious to to just expand how things work with the "same rules" when complex numbers allow you to do things that weren't possible before they were introduced. The beauty is it having all make sense. If you introduce something new and it breaks something else and then you need to patch that... you get modern game balance teams. But in all seriousness, I think one needs to be quite smart and creative to be able to innovate instead of derive, which, to g8ve credit, can sometimes be quite innovative in its own right lol I mean the idea of complex numbers is just an algebraic closure of the real numbers. Complex numbers are older than the fundamental theorem of algebra. I don't really know what "when complex numbers allow you to do things that weren't possible before they were introduced" means. Complex numbers were discovered because we were trying to solve cubic equations. The beauty is it having all make sense. If you introduce something new and it breaks something else and then you need to patch that Again I just don't understand what this means. Are you suggesting that the mathematical results derived from LLMs so far are being falsely verified by mathematicians? What part of mathematics are you suggesting that is has literally a single iota of relevance to? But in all seriousness, I think one needs to be quite smart and creative to be able to innovate instead of derive, which, to g8ve credit, can sometimes be quite innovative in its own right lol This just feels like a goofy sort of philosophical argument. How do you define innovate and how do you define derive? It feels like you're saying complex numbers were 'innovated' when you could easily argue (and I'm being particularly non-rigorous here) that they were derived from having proven solutions for specific cubic equations and then being able to demonstrate that those equations were also equal to simpler equations of real numbers and negative square root numbers (i.e. complex numbers). | ||
|
Manit0u
Poland17813 Posts
On August 01 2026 01:39 Jankisa wrote: I guess we have a very different interpretation of what is impressive and how innovation works. Even then, you consistently skip over the examples I provided with AI systems coming up with novel ways of playing the games of Chess, Go and Starcraft 2, to varying success. Also, your quote about the HF incident didn't really say anything except what I noted when I first posted about it, that OpenAI and other labs are very lax with their controls. The AI (and Mythos and other systems) are by far, from all human activities biggest experts in coding, and when tasked with breaking systems they do find novel exploits, you asserted that they used known ones without any proof or source, when I asked for it you provided a quote that does not support what you said. Asking for some "breakthroughs that no one saw coming" in the age where these are incredibly rare and have been mostly incremental and "group projects" for us humans is in my opinion, at this point especially kind of silly. LLMs went from having trouble solving extremely easy math problems 2 years ago to solving things that 80 years of mathematicians trying couldn't, it's weird to dismiss them. I'm sorry but finding new way to play chess isn't really impressive in my eyes. It's not the most complex game out there and like it was mentioned it can be brute-forced since there are no random factors involved. Computers have been beating people at chess way before AI. As to not providing the source for the bugs I don't really have time to go through the network security articles and videos I've been through lately to find where it was mentioned because it's also mostly inconsequential. And the claim that AI system are "by far biggest experts in coding" is actually laughable. I work as a senior software engineer and I see what AI can and can't do on a daily basis. If you're writing a blog post or a to-do app sure. But if you need anything sufficiently complex and something that you want to be able to collaborate on with others and maintain in the future AI falls woefully short of expectations. It's ok as a tool to help you analyze a big codebase you're not 100% familiar with yourself and give you some hints about how things work but it is itself unable to produce code that would be up to the quality and standards required for bigger projects. | ||
|
Jankisa
Croatia1559 Posts
On August 01 2026 02:50 Manit0u wrote: As to not providing the source for the bugs I don't really have time to go through the network security articles and videos I've been through lately to find where it was mentioned because it's also mostly inconsequential. And the claim that AI system are "by far biggest experts in coding" is actually laughable. I work as a senior software engineer and I see what AI can and can't do on a daily basis. If you're writing a blog post or a to-do app sure. But if you need anything sufficiently complex and something that you want to be able to collaborate on with others and maintain in the future AI falls woefully short of expectations. It's ok as a tool to help you analyze a big codebase you're not 100% familiar with yourself and give you some hints about how things work but it is itself unable to produce code that would be up to the quality and standards required for bigger projects. Let's do it step by step. OK, you aren't impressed by Chess, how about Starcraft? Is that also a very easy game? Since you are a programmer, you should also know the difference between the Deep Blue way of bruteforcing chess (which is ironically how you seem to understand all AI it seems) and what Stockfish is doing. On the HF incident, come on man, that is such a cope out, it's fine to admit that you made shit up. There is no source for what you claimed (specifically that AI used 2 known exploits for a different system) because you made it up, if you didn't you could just put that sentence in AI and find the source in 30 seconds, I mean you could also find it in your browser history, if it was real. As a senior Sysadmin I can tell you that AI finding 2 zero day exploits just to break containment over a weekend is not inconsequential, it's impressive as fuck. I also did not say that AI's are "by far biggest experts in coding", I said that from all the things that they have been trained to do, they are excelling or the biggest experts in specifically coding, not math, not physics, coding. I've heard this excuse you are using on how AI is not that good for programming from senior developers that are falling way, way behind guys who are actually embracing AI in my company, the difference between them and their peer who does use it is staggering, and we work on a fairly mature product with a huge code base. As long as you know how to approach it, segment it and focus the AI to do exact things, it's amazing. The most senior guy, the boss of both of the 2 guys I referenced above said AI writes about 80 % of his code, and I believe him, because he knows how to use it and the results are spectacular and fast. | ||
| ||
