Submitted 7 hours ago by salacious_coaster@feddit.online to [deleted]
The Internet being mostly broken at this point is driving me a little insane, and I can’t believe that people who have the power to keep a functioning Internet for themselves wouldn’t go ahead and do it.
I wonder about this every time I see people(?) crowing about how amazing AI is. Like, is there some secret useful AI out there that plebs like me don’t get to use? Because otherwise, huh?
I assume all high-level positions in tech companies are using better versions than what they shove out to the rest.
I mean, Microsoft treats Enterprise users with class with Windows 10 Enterprise. That version doesn't have nearly the amount of bloat that even Professional has. Hell, Enterprise doesn't even have that stupid online search function.
I dont think any of these tech execs (all execs?) use their products. They all have assistants to do everything for them, so they have no idea what this whole “internet” thing is, other than it makes them money.
Oh they know how to get to the porn.
No, porn is for poor people. These people can (and do) easily afford the live action version.
Pretty sure they don’t bother with just photos, I’m sure there’s some guy that’s replaced the one that died in prison.
They probably use exclusive paid sites. Maybe even … very exclusive.
Yeah, they just said that their assistants handle everything. Little bit of mood lighting, little bit of a new BMW, little bit of a cover up and hush money. All expenses to the company. Ahh, the good life.
Yes, actually.
The “preview” version are often pretty good, before Google deep fries them with sycophantic RHLF. For example, Gemini 2.0 and 2.5 Pro both peaked in temporary experimental versions, before getting worse (and benchmark maxxed) in subsequent updates.
Those assholes probably kept a working version of Inbox for themselves. 😡
What did inbox improve on?
There is Kagi for the rest of us
Yeah this is what I was thinking, they use Kagi or something like it.
I wish they also had an LLM. I need a paid backup for when ChatGPT enshittifies
They do offer access to ALL the LLMs. You could pay $20 to OpenAI for ChatGPT 5, or you could pay $25 to Kagi for unlimited Kagi search and access to premium models for ChatGPT, Claude, Google, Mistral, Kimi, and others.
The problem with conspiracy theories like this is it assumed that the people doing the conspiracy are competent.
every time I see people(?) crowing about how amazing AI is
You’re correct that there’s a massive flood of bots pushing it everywhere. But regardless of what the subject is, once someone has “bought in” to a scam they tend to stick with it and defend it no matter what. Because the alternative is admitting they were fooled, and that’s basically an uncrossable bridge for most folks.
People on their literal death beds were using their literal last words (before being intubated with covid) to threaten nurses not to go near them with “the jab”. So it really doesn’t surprise me that people continue using “AI” despite it being worse than worthless for literally everything
people continue using “AI” despite it being worse than worthless for literally everything
On the contrary, AI’s been extraordinarily useful for me this year. BUT– I try my best to understand its ins and outs… i.e., where it’s most accurate, and when it’s most prone to hallucinating confidently incorrect replies.
Pretty much any tool has a narrow range of uses, and is useless for everything else. So it kinda makes sense to me that a ‘do all’ tool would naturally have plenty of flaws in its early stages.
“One of the saddest lessons of history is this: If we’ve been bamboozled long enough, we tend to reject any evidence of the bamboozle. We’re no longer interested in finding out the truth. The bamboozle has captured us. It’s simply too painful to acknowledge, even to ourselves, that we’ve been taken. Once you give a charlatan power over you, you almost never get it back.”
― Carl Sagan
Yeah, that’s probably the case. But hell, the bots are working. They got me second-guessing myself and wondering if I just haven’t seen the “good” AI that the elites are keeping for themselves.
This is really the fear we should all have. And I’ve wondered about this specifically in the case of Thiel, who seems quite off their rocker.
Some things we know.
Architectural, the underpinnings of LLM’s existed long before the modern crops. Attention is all you need is basic reading these days; Google literally invented transformers, but failed to create the first llm. This is important.
Modern LLM’s came through basically two aspects of scaling a transformer. First, massively scale the transformer. Second, massively scale the training dataset. This is what OpenAI did. What google missed was that the emergent properties of networks change with scale. But just scaling a large neural network alone isn’t enough. You need enough data to allow it to converge on interesting and useful features.
On the first part, of scaling the network. This is basically what we’ve done so far, along with some cleverness around how training data is presented, to create improvements to existing generative models. Larger models, are basically better models. There is some nuance here but not much. There have been no new architecural improvements that have resulted in the kind of order of magnitude scaling in improvement we saw in the jump from lstm/GAN days, to transformers.
Now what we also know, is that its incredibly opaque what is actually presented to the public. Open source models, some are in the range of 100’s of billions of parameters Most aren’t that big. I have quen3-vl on my local machine, its 33 billion parameters. I think I’ve seen some 400b parameter models in the open source world, but I haven’t bothered downloading them because I can’t run them. We don’t actually know how many billion parameters models like Opus-4.5 or whatever shit stack OpenAI is sending out these days. Its probably in the range of 200b-500b, which we can infer based on the upper limits of what can fit on the most advanced server grade hardware. Beyond that, its MoE, multiple models on multiple GPU’s conferring results.
What we haven’t seen is any kind of stepwise, order of magnitude improvement since the 3.5-4 jump open AI made a few years ago. Its been very… iterative, which is to say, underwhelming, since 2023. Its very clear that an upper limit was reached and most of the improvements have been around QoL and nice engineering, but nothing has fundamentally or noticeably improved in terms of the underlying quality of these models. That is in and of itself interesting and there could be several explanations of this.
Getting very far beyond this takes us beyond the hardware limitations of even the most advanced manufacturing we currently have available to us. I think the most a blackwell card has is ~288GB of VRAM? Now it might be at this scale we just don’t have hardware available to even try and look over the hedge to see what or how a larger model might perform. This is one explanation: we hit the memory limits of hardware and we might not see a major performance improvement until we get into the TB range of memory on GPU’s.
Another explanation, could be that at the consumer level, they stopped throwing more compute resources at the problem. Remember the MoE thing? Well these companies, allegedly, are supposed to make money. Its possible that they just stopped throwing more resources at their product lines, and that more MoE does actually result in better performance.
In the first scenario I outlined, executives would be limited to the same useful, but kinda-crappy LLM’s we all have access to. In the second scenario, executives might have access to super powered, high MoE versions.
If the second scenario is true and when highly clustered, llm’s can demonstrate an additional stepwise performance improvement, then we’re already fucked. But if this were the case, its not like western companies have a monopoly on GPUs or even models. And we’re not seeing that kind of massive performance bump elsewhere, so its likely that MoE also has its limits and they’ve been reached at this point.
If a small group of people get to outcompete the rest of society thanks to having exclusive access to more powerful LLMs, I’m sure (🤞) that would lead to a lot of unrest. Not only from us plebs, but also from the rest of business. I can see it leading to some lawsuits concluding that AI advancement must be shared (if at a fee) with the public.
Don’t forget the fundamental scaling properties of llms, that openai even used as the basis for strategy to make chat gpt 3.5.
But basically llm performance is logarithmic. It’s easier to get rapid improvements early on. But at later points like we are now require exponentially more compute, training data, and model sizes to get now small level of improvements.
Even if we get a 10x in compute, model size, and training data (which is fundamentally finite), the improvements aren’t going to be groundbreaking or solve any of the inherent limitations of the technology.
Diminishing returns
💯
Scaling is a quick and dirty way to get performance improvements. But there is no guarantee that we get any more interesting behavior or don’t get like, wildly more interesting behavior with an additional 10x’ing. The fact is we simply don’t know what emergent properties might exist at a networks size we simply phsyically can’t scale to right now. Its important to not assume things like “its just going to be diminishing returns”, because while that is most likely the case, precisely this thinking is why google wasn’t the first to make an LLM, even though they had discovered/ invented the underlying technology. Yet another 10x scaling didnt result in just diminishing returns, but fundamentally new network properties.
And that principal holds across networked systems (social networks, communication networks, fungal and cellular communication networks). We truly do not know what will result from scaling the complexity of the network. It could move the needle from 95% to 96.5% accuracy. Or it could move it to a range of accuracy that isn’t measurable in human terms (its literally more accurate than we have the capability of validating). Or it could go from 95% to 94%. We simply don’t know.
Its also possible we’ve reached the limits of the training data.
This is my thinking too. I don’t know how to solve the problem either because datasets created after about 2022 likely are polluted with LLM results baked in. With even a 95% precision that means 5% hallucination baked into the dataset. I can’t imagine enough grounding is possible to mitigate that. As the years go forward the problem only gets worse because more LLM results will be fed back in as training data.
If model collapse is such an issue for LLMs, then why are humans resistant to it? We are largely trained on output created by other humans.
I mean thats possible, but I’m not as worried about that. Yes it would make future models worse. But its also entirely plausible to just cultivate a better dataset. And even small datasets can be used to make models that are far better at specific tasks than an any generalist llm. If better data is better then the solution is simple: use human labor to cultivate a highly curated high quality dataset. I mean its what we’ve been doing for decades in ML.
I think the bigger issue is that transformers are incredibly inefficient about their use of data. How big of a corpus do you need to feed into an llm to get it to solve a y =mx+b problem? Compare that to a simple neural network or a random forest. For domain specific tasks they’re absurdly inefficient. I do think we’ll see architectural improvements, and while the consequences of improvements has been non-linear, the improvements themselves have been fairly, well, linear.
Before transformers we basically had GAN’s and LSTM’s as the latest and greatest. And before that UNET was the latest and greatest (and I still go back to, often), and before that basic NN’s and random forest. I do think we’ll get some stepwise improvements to machine learning, and we’re about due for some. But its not going to be tittering at the edges. Its going to be something different.
The only thing that I’m truly worried about is that if, even if its unlikely, if you can just 10x the size of an existing transformer (say from 500 billion parameters to 5 trillion, something you would need like a terabyte of vram to even process), if that results in totally new characteristics, in the same way that scaling from 100 million parameters to 10 billion resulted in something that, apparently, understood the rules of language. There are real land mines out there that none of us as individuals have the ability to avoid. But the “poisoning” of the data? If history tells us anything, its that if a capitalist thinks it might be profitable, they’ll throw any amount of human suffering at it to try and accomplish it.
Thanks for the detailed answer!
I fully believe in old.Google.com existing now
Old Google was the tits. Even five years ago Google would be hyper advanced technology today.
Wake up, babe. New conspiracy theory just dropped.
I choose to believe this now.
Right? I know this is a conspiracy theory for which I have no evidence. It just makes too much sense to me to not believe it.
I’m not sure it makes sense, but I ❤️ the idea that there’s a functional search engine out there. AI’s best advantage right now is that its the first innovation for searching the web to drop in a LOOOOOONGtime.
Microsoft employees are the beta testers of their software. I expect no different from Google.
It doesn’t take a lot of tech skill to be an exec for a tech company. My guess is they fall into the same category as those that see the AI overview on Google and think “wow this is so much better” and never second guess the results.
When they unveil it to the public, it will be subscription-based.
Google keeps a secret un-enshittified version of their search engine for anybody to use. Instructions here: tenbluelinks.org
This removes AI but does not undo pragavar rhagavan’s changes
Apple execs use iPhones with a modified iOS that lets them use a keyboard that doesn’t suck and doesn’t have sponsored auto-corrections, and I will die on this hill. /s
Honestly though, I think a lot of people at Microsoft just use Macs. Google? I have no idea. I would imagine the smarter ones search with DuckDuckGo or something like that. Apple? They probably just use their own stuff since they look down their noses at the competition, but they use the best. Co-founder Steve Wozniak basically confirmed that. Even though he no longer works there, he said they send him the best iPhone every year. (I’m not sure if they still do that.) He also said it’s pretty but he wishes it did half the shit his Android phone does. He also uses custom firmware on Android, so he’s not just using a stock Android phone, he’s running something like GrapheneOS or LineageOS. So Woz is basically just collecting them at this point. Maybe he donates them. I dunno. So yeah, Tim Cook has a Mac on his desk, but it’s not the $480 M4 Mac mini, it’s a fully spec’d Mac Studio that would probably set you back close to 10 grand. Because why would he use anything less? And he’s carrying the top spec’d MacBook Pro. And a 1TB iPhone Pro… he doesn’t seem like a big guy, so he might just be using the “regular” Pro and not the Pro Max, as a personal preference. Honestly, the executives can use whatever they want, but they’ll only be seen with the flagship.
I worked for a FAANG company for a lotta years. We always had the fully spec’d laptop/desktop/phone. Its not reserved for just the C-suite. The tech is cheap compared to almost every other aspect of employing expensive labor. Hell, the food we got every day probably cost the company well more than the tech.
Honestly, I wouldn’t be surprised if there were some secret Google search engine that still has decent search results instead of the paid-for mess that us plebs get.
Gemini 3 Pro is pretty nuts already.
But yes, labs have unreleased higher cost models. Like the OpenAI model that was thousands of dollars per ARC-AGI answer. Or limited release models with different post-training like the Claude for the DoD.
When you talk about a secret useful AI — what are you trying to use AI for that you are feeling modern models are deficient in?
what are you trying to use AI for that you are feeling modern models are deficient in?
Giving correct answers more than half the time.
Can you give an example of a question where you feel like the answer is only correct half the time or less?
The short answer is yes, and most companies do.
I have a small consultancy with a few staff, and im passionate about self hosting.
Using FOSS projects I’ve configured a selection of private services, which includes search and Gen AI. If I can do this with no resources…
Never considered self-hosting search. How does your LLM compare to the public ones?
Not completely self hosted. Searxng is just a front end for a bunch of other engines, so it fetches results from google et al. Its good but glitches a lot as the engines themselves try to block it.
Yacy is self hosted, it crawls the sites you tell it to.
The LLMs are great. You can self host the models or just link to the hugging face API. The models you can find there are tuned for specific purposes and are far superior for those purposes. You can also train “agents” on top of those models, like upload your policy and procedure manual and ask it to create additional procedures in that context.
They haven’t released the best AI because it’s too valuable
QuinnyCoded@sh.itjust.works 10 minutes ago
another thing to mention, YOUTUBE. The search bar doesn’t even do anything, it shows RECOMMENDATIONS instead of answers to the search.
Paying doesn’t even stop that! It’s actually maddening