It’s not the crawling that is the issue. Facebook for example shows up just fine on search results, you just need an account to view the content on FB if it isn’t public. User privacy settings also shield content. Google has actively and willingly destroyed its utility as a search engine by placing revenue sites/preferred results ahead of actual search results. What they want you to $ee comes first, and if social media like facebook isn’t profitable to google, you don’t get to see it. I did a quick search for “facebook car groups” and got a page full of them, so you absolutely can get those walled gardens in results.
Comment on Has google stopped working for finding anything?
Deftdrummer@lemmy.world 11 months ago
The long and short of it - Google search was designed at a time when the web was in its infancy. Basically just text and a few images.
Fast forward to today, and reddit is the only one that still allows its data to be crawled.
As media has become more social (basically all of it) the walled gardens prevent you from even viewing content without an account.
Every platform wants you to be searching inside their service.
Google is useless.
RememberTheApollo_@lemmy.world 11 months ago
Russty@lemmy.world 11 months ago
Facebook shows up, but do any of the conversations with meaningful information show up in a search?
Unfortunately I think this will get worse over time as databases of text and video are extremely useful for LLM training, and locking them down makes sense if you look at them as an asset that can be licensed.
Zomboomafoo@slrpnk.net 11 months ago
That explains why site:reddit.com continues to be the best way to use Google, I thought I was just stuck in my habits
Goodtoknow@lemmy.ca 11 months ago
What’s upsetting to me is how many communities that have moved to discord and there’s no centralized way to search for content in there. Black hole of ethereal content
tslnox@reddthat.com 11 months ago
And the Discord search is total garbage. There’s no way (I know of, and I’ve tried to search for it) to search for the exact term. No parentheses, no plus signs or whatever. So you try to search one term, but the results come back full of terms that are similar but not the same.