Welcome to DU!
The truly grassroots left-of-center political community where regular people, not algorithms, drive the discussions and set the standards.
Join the community:
Create a free account
Support DU (and get rid of ads!):
Become a Star Member
Latest Breaking News
Editorials & Other Articles
General Discussion
The DU Lounge
All Forums
Issue Forums
Culture Forums
Alliance Forums
Region Forums
Support Forums
Help & Search
Google AI and ChatGPT AI are Easily Manipulated By Bogus Reddit Posts
This report comes from 404 Media. They focus on the mods in a subreddit but AI is scraping all the utter nonsense on Reddit and treating it as facts:
In a post last week, the moderators of r/biohackers said they would be banning new posts about peptides and hormone replacement therapy (HRT) because of attempted manipulation by the companies that make, market, and sell them. [...] "As AI search engines increasingly pull answers from Reddit, companies are using us for AEO. On top of that, there's been an explosion of peptide interest and AI usage flooding the sub. Together, this has put serious pressure on content quality," a post by the moderators read.
...
It has become incredibly difficult to stop Reddit manipulation, because the firms doing it are getting more sophisticated. The moderator said that there are really standard and long-running strategies where brands will hop in the comments and suggest their products: "That type of marketing has always existed and if people want to try something new because the brand resonated with them, cool. That's the way marketing should flow in my mind," they said. "But what I'm seeing that is way scarier to me is that there are companies that will reverse-engineer the actual prompt patterns that are prioritized by LLMs, and so you'll see someone post a super clickbait, high-traction, vague question like 'Is all the hype around Vitamin D actually worth it?" they added. "And that thread will do really well because everyone on biohackers actually has an opinion, so it gets engagement and prioritized by LLMs, and then brands will sneak in and they'll embed their brand mentions in those threads in the exact right places in a seemingly organic way. But none of it is organic, the entire thing is a strategy by an agency to prioritize brand mentions or a narrative within an LLM."
The Reddit accounts that are doing this are "warmed up" or are made to seem human, meaning they have a posting history that is not just promotional. This makes them much harder to detect and moderate against. Some of the agencies doing this are paying real people to post promotional content, or have built communities where people are incentivized to post promotional content. The moderator said that Reddit's automated moderation tools have been helpful, but that the type of promotion happening has become so sophisticated that it has become more of a you-know-it-if-you-see it kind of thing. "A lot of it has become pattern recognition," they said. "You literally just sort of know what to look for. But the problem is you don't want to become punitive to the people who aren't doing this maliciously, and so I think the over-moderation risk is very real."
https://www.404media.co/companies-are-using-reddit-to-manipulate-chatgpt-and-google-ai-search/
...
It has become incredibly difficult to stop Reddit manipulation, because the firms doing it are getting more sophisticated. The moderator said that there are really standard and long-running strategies where brands will hop in the comments and suggest their products: "That type of marketing has always existed and if people want to try something new because the brand resonated with them, cool. That's the way marketing should flow in my mind," they said. "But what I'm seeing that is way scarier to me is that there are companies that will reverse-engineer the actual prompt patterns that are prioritized by LLMs, and so you'll see someone post a super clickbait, high-traction, vague question like 'Is all the hype around Vitamin D actually worth it?" they added. "And that thread will do really well because everyone on biohackers actually has an opinion, so it gets engagement and prioritized by LLMs, and then brands will sneak in and they'll embed their brand mentions in those threads in the exact right places in a seemingly organic way. But none of it is organic, the entire thing is a strategy by an agency to prioritize brand mentions or a narrative within an LLM."
The Reddit accounts that are doing this are "warmed up" or are made to seem human, meaning they have a posting history that is not just promotional. This makes them much harder to detect and moderate against. Some of the agencies doing this are paying real people to post promotional content, or have built communities where people are incentivized to post promotional content. The moderator said that Reddit's automated moderation tools have been helpful, but that the type of promotion happening has become so sophisticated that it has become more of a you-know-it-if-you-see it kind of thing. "A lot of it has become pattern recognition," they said. "You literally just sort of know what to look for. But the problem is you don't want to become punitive to the people who aren't doing this maliciously, and so I think the over-moderation risk is very real."
https://www.404media.co/companies-are-using-reddit-to-manipulate-chatgpt-and-google-ai-search/
To me the problem is AI is scraping info that is not fact checked. The two biggest sources for ChatGPT and Google AI answers are Reddit (~40%) and Facebook (~27%). Of course they are being manipulated. They have made the mods' jobs impossible. But what about the millions of posts that are just hot takes and misinformation; people not trying to monetize their misinformation, just being mediocre and not expecting to have their baseless opinions repackaged as 'must be true because AI said so.'
The UK just made Google AI include links with their answers and let responsible and accurate publisher opt out. This may lead to even more bias toward anonymous Reddit users and bots.
Chat j'ai pété indeed.
3 replies
= new reply since forum marked as read
= new reply since forum marked as read
