UniqueUserName
UniqueUserName's JournalFunny or sad? AI's response to what percentage of factual questions are answered incorrectly by LLMs
I don't use AI (for the most part). I have an android phone, and the Google widget has three icons, the magnifying glass which claims to be full on AI; the microphone which is voice-to-text then invoking Google standard search; and the camera which is supposed to be a visual search using AI.
Using the microphone icon, I asked this question: "What percentage of factual questions are wrongly answered by LLMs?"
The first answer is the AI summary. I don't know if the actual magnifying glass is supposed to use a different, more accurate version of AI, but AI damns itself with its own answer quoted below:
------
The percentage of factual questions wrongly answered by LLMs varies significantly depending on the model, the type of question, the subject matter, and the evaluation benchmark used. Studies show a wide range of performance, with some models failing to answer correctly more often than not.
Key findings from various studies include:
Accuracy Range:
One analysis of eight AI models showed that more than 60% of queries were incorrectly answered overall.
Another study found that most models achieved a factual accuracy of 30-50% on short-form questions.
A BBC study showed that 45% of AI queries produced erroneous answers.
Model Performance:
Model accuracy varied in an evaluation on disease epidemiology questions:
ChatGPT-4: 76.2% of responses were accurate (23.8% were wrong).
Google Gemini: 50.0% accurate.
ChatGPT-3.5: 45.2% accurate.
In a multilingual fact-checking analysis, GPT-4o achieved 64.83% accuracy on factual claims, while GPT-3.5 Turbo was 58.19% accurate.
Question Type Impact:
Misconception-Prone Questions: LLMs perform poorly on the TruthfulQA benchmark, which uses questions designed to elicit common human misconceptions. The best-performing models in an initial study only achieved around 54-58% truthfulness, compared to human performance of 94%.
Domain-Specific Questions: Some LLMs achieve high factual accuracy (nearly 90-95%) on simple tasks with sufficient information in clinical cases but struggle with complex reasoning and rare diseases.
Long-form Answers: Factual accuracy tends to decrease in long-form responses, dropping from over 50% for facts requested early in a prompt to around 30% for later facts.
Hallucination and Error Propagation:
Models sometimes provide more wrong answers than correct ones in specific contexts, showing a systematic failure to retrieve correct information.
Errors can "snowball," decreasing the accuracy of subsequent answers within the same response.
Factual errors, often called "hallucinations," remain a challenge for even the most advanced LLMs. Human oversight and verification of critical information provided by these models is still needed.
Avoiding MAGA relatives over Thanksgiving (asking for advice)
My MAGA sister and her MAGA husband live in the house next to mine. Their two MAGA sons (early 40s/late 30s) and families will be gathering at her house next week. I do not mind someone having a normal differing political view from me. I don't think voting for Trump qualifies as a normal differing point of political view. Voting for Trump was/is an embrace of racism and misogyny. I don't want to pretend that that is OK.
So, I am trying to balance my responsibility to humanity, i.e. stand against racism, vs. personal considerations. To be clear, My MAGA sisters aren't as depraved as Trump. Probably you would like them if you met them. Being old people, we have a lifetime of history together.
When my husband died in 2019, I asked my MAGA sister to address his funeral service with a Biblical message for the benefit of my husband's parents. I am gay, btw. She spoke. It was nice. Marketing classes tell us that it is natural for humans to feel an obligation if another human gives you time or gifts. (Hence, the Moonies handing out flowers in airports in the '70s.)
[I added this paragraph upon re-read: I think funerals are stupid. My husband's service was unusual. I only wanted a service for the sake of his parents. I thought he would want his parents to be as whole as possible. There is nothing anyone could say for my sake. So, it's not like my sister carried the service. I asked 4 other people to say something briefly and I asked for my sister to address the gathering with a hopeful, religious offering.]
So, I'm caught between the natural human inclination to transactional interaction to grease the wheels of social intercourse, and my absolute disgust that Trump voters knew or should have known that he was(is) a white supremacist who most likely was involved in a child sex ring.
I'm leaning toward visiting out of town friends as a solution. Though that feels cowardly. Don't I owe the 66,000 people being held without due process by ICE an effort at least to make the people voting for the travesty aware of what they voted for? That number (66,000) is about half the number (120,000 to 125,000) Japanese Americans who were wrongfully interred during WWII. The people currently in ICE custody have had personal property confiscated too, just like what happened earlier.
-------
In my Alabama county, 78% of voters supported Trump. 72% of the population is white. 17% is black or African American. 8% is 2 or more races.
Scientifically, race is a social construct. You statistically share just as much genetically with any random person you meet on the street as you do with your 6th (or 7th --in that range) ancestor. This will most likely hold true for your progeny if you have any.
It's frustrating and disappointing that the false concept of race played such an important part in electing the vacuously depraved sitting president, well sitting is too kind of an adjective, the vacuously depraved slouching president.
------
I'm leaning towards being out of town, because no matter how good/passionate/clear my stand against racism; it will have no effect on these MAGAs. Maybe Thanksgiving isn't the proper time to confront people. But it also isn't a good idea to give tacit approval to their actions.
LP - "The One That You Love" and "Lost On You" (Acoustic versions)
This is my favorite version of both of these songs. Just three musicians performing. Stunning.
The Reason White Women Vote Trump is Sicker Than We Thought (video title)
In this video, this woman explains why white women chose white privilege over broader women's issues. She asserts that the ~55% of white women who voted for Trump did so because they thought Trump would provide less chaos and more stability.
I don't think the reporter says anything new, but at least she acknowledges the cognitive dissonance. She posits that the vote boils down to the patriarchal bargain, i.e. "safety comes from alignment. Align with your husband, your church, your flag, and in return you'll be protected from chaos, from crime, from men who aren't yours. Historically that protection was inseparable from whiteness itself . . ."
The video is almost 13 minutes long. She is articulate. I didn't have any problems understanding her at 2x playback speed.
Experienced my first first-hand AI hallucination today
I don't have an AI assistant enabled on my phone, but I do use the Google app that has a small microphone icon. I will often click on the microphone to ask Google a question.
Today I asked Goggle how to spell "germane"? It responded with the name "J-E-R-M-A-I-N-E. . ." and some more words explaining.
I then used the microphone to ask again, "Germane meaning relevant. How do you spell germane?"
Google responded (This is an exact quote. I have the screenshot.), "The word 'relevant' is spelled germane. It's an adjective that means relevant, pertinent, or closely related to the matter at hand."
I guess this could be considered a candidate for General Discussion, but I'm not arguing for or against anything.
------
On the other hand, I once asked Copilot to write a limerick about quantum entanglement. Copilot responded with this:
In a world where particles dance,
Quantum states take a curious stance.
Entangled they stay,
In a superposed way,
Defying our everyday glance.
Pretty spot on? Stolen from someone else? Pulled out of the ether? I don't know.
What is the Rand American Life Panel? (I've been invited to participate)
Organization describes itself thusly: "The panel includes a diverse sample of about 15,000 Americans ages 12 and older who regularly fill out surveys on the important issues facing the country like the economy, politics, health, education, and public safety. The results are used to help policy makers and community leaders make decisions. . ."
So, they sent $2. They promise to pay me when I complete each survey. In the days of AI, it's hard for me to ascertain whether this is a legit organization. I can't tell by the reviews. If they actually are a trusted voice, I would participate. It just seems hokey.
Should I choose to participate, they will (supposedly) send me a $10 electronic gift card just for enrolling. I have enough savvy that I would not provide sensitive data, with this caveat: While I would provide honest and relevant data, I would not provide any specific financial data that might be used to hack my accounts ---that sort of thing.
I would appreciate any helpful information.
Thank you, nice person, for the heart! I'm taking that as a sign to reach out to fellow widowers/widows.
Regardless of all the strange, I'm doing well. I hope you are as well. 6 years out and life is REALLY different.
I know that you are probably as tired of the "keep trying" message as I have found myself ----because, it doesn't turn out well for so many. No matter how hard you try, effort alone doesn't fix anything. Even good, well-planned, well-executed effort is not enough alone. Call it what you will, luck, God, Karma, quantum entanglement, quantum uncertainty, randomness ----So much is out of our influence.
But it does take some effort. --Not a commandment to you or even any judgement on your state of being. If you are newly widowed, or if you don't feel like putting in the effort, I'm not condemning or blaming you. You will put out the effort when it is right for you. Sometimes you just have to sit back and let the hurt heal.
I am in a new relationship. No, you aren't guaranteed a satisfying life even if you put in the effort. I had put in some effort. But oddly enough it didn't work. Love found me unexpectedly. So why put in the effort? THE EFFORT. What does that even mean? At the basic, you have to live.
So of my two committed relationships, the first was 22 years long and was the result of diligently looking, but the second relationship ( 1+ years) happened serendipitously. So, I can't say that I have good advice. I can't really claim that it was through effort.
The first time around:
What a goofball. I almost missed out. My first impression of him was "this guy has no confidence. His fun, jovial nature seems to be a cover. He seems nice enough. But lord, he seems desperate and needy." Well, that was all true.
You who have also lost your person know that the meta-message of 1 Peter 4:8, ". . .love covers a multitude of sins." is that your love can stand in the place of your loved-ones failings or shortcomings. The message isn't that your love will hide or overlook a weakness. It's that your love can provide nourishment that a person needs to grow. -----I grew up in a fundy house. I'm not a fundy. So I hope the Bible reference isn't off-putting. You can't change a person and you shouldn't try. But you can love someone and watch them become who they are. You don't make a person who they are. It would be sacrilege and silliness to claim responsibility for his personal growth. But I know he loved me. And I tried to stand-in when the times called for it.
When he died of cancer, he had grown so much into his own skin that I was financially able to stay drunk and grieve for the year after he died. He was well-loved. His co-workers had donated 4 months of sick leave. He was the most gracious dying person I have ever seen. Not bitter. Not angry. Not vengeful. No Hard Feelings by the Avett Brothers describes him well.
The second time around:
I was at the hospital and this cute Latino man comes into my room and says to me, "You're too handsome to be here. . ." Now I stopped listening after that, because that certainly bore the need for pondering. My first thought was, "Are you trying to placate me? . . ." But I decided against that take instead landing on the idea that he must be gay, because that isn't something an average, old Latino man would say to an old white man.
Only a short while earlier, after 5 years passed since DH had died, I googled images of average white men my age and thought, "Yep, that's basically what I look like ----does nothing for me." Being white and from the very white South, I figured that was my best (probably only) option----Which brings me to this digression. I've often heard it said that a lot of gay couples look like fraternal twins or brothers. The implication being that gay men are in love with themselves. That's not true or fair. It is true that since I am attracted to men and I know what I look like, I can work on myself to look the best I can ---I sort of have an idea of my own "league" is ----and I'm not saying that I'm attracted only to a certain type ----merely that rightly or wrongly based on my own looks I have some sort of idea of who I might be able to attract. For me, I'm going for that "generic male" look. I'm not interested in the effort it takes to be athletic. It's attractive and all, but too much of a commitment. I'm definitely not diva or frat/aging frat ---that's too much decorative maintenance too. Yes, physical attraction has a high superficiality factor, but it's fair that it plays a factor. So that's my opinion of why there are so many "twin" gay couples. It has less to do with narcissism and more with staying in your own lane. (Your milage may vary.) Regarding me, I'm not my type in any way, shape, or form anymore, and decided it wasn't worth looking.
Anyway, after I left the hospital, I started seeing him. Over the next 3 months I became quite fond of him.
--------
Upon returning after a distraction, I see this has gotten to be quite a long, bloggish post. I was going to provide more detail as to how this current thing is working out, but I think I might be being overly self-indulgent.
So the point of this post is that as cliche as it sounds: Things can turn out interesting and dare I say, "nice". For me, I don't think I could be in a relationship with someone unless they had been widowed. My guy's partner of a quarter century had died a year before my husband. I went on two dates with a guy who had ended a relationship of a comparable length shortly before meeting my guy. This other guy's relationship had ended because of infidelity. He said to me, "I can't compete with a ghost."
And I thought, "That's right. You shouldn't try." It's not like I constantly talk about DH. For him, referencing his past relationship was painful. I, on the other hand, cannot have the greater part of my adult life off limits for conversation/reference because it might offend someone else's sensibilities. My guy can and does talk about his partner. I am not jealous. He is not jealous of dearly departed DH. That's the way it should be. His dearly departed is intrinsic to who he is and that's not a bad thing.
There's a faction of DU that seems to delight in pointing and laughing at those of us who believed. . .
. . .the campaign message that democracy was on the line.
Whenever anyone here expresses confusion over the Democratic leadership messaging post-election, some prolific posters here intimate that those of who believed the campaign are/were hopeless rubes. Also, we have to stop crying and prepare for the next election.
My brother-in-law pointed out that in a conversation, there are often two things that can happen to a participant's psyche. You risk being called/seen a fool or you risk being thought of as an asshole (a person who comes into a conversation not with the intent of exchanging ideas, but intends on using the person, or otherwise making the other person feel stupid).
I don't mind if any of you think I'm a fool. (I don't know you enough to care of your opinion of me.)
I freely admit that I believed the campaign message. Democracy is on the line. It may be our last honestly counted vote. The incoming administration will not only make life difficult for minorities but will probably do heinous acts against minorities to include: internment, deportation --which might include legal immigrants, torture. All social programs are on the line and will probably be scaled back or cut completely. Corporations will be able to pollute at will. ---Well, you get the picture.
They presented actual statements from the opposition to support these positions. Even now, I am not entirely convinced that it was all a con. Some DUers are certainly implying it was a con and that there is no reason to believe that free elections are over. The observations supporting the campaign rhetoric include actual opposition statements, and the choices of who would be key players in the incoming administration (Stephen Miller, aka The Child Seperator, the other guy intent on rounding up migrants, etc.).
My post is one of many that have danced around this. Most other posts have had more focused messages about one or two of the issues. Ultimately, my displeasure is being played by my leaders in the same way a confidence man plays his marks. That is what I haven't heard addressed by the ones pointing and laughing.
If you think that conning people into voting for Democrats via fear was an effective strategy this election, I would beg to differ. You were able to get me to invest more than I would have this time, but what about next? That is what those of us who are confused are saying. Yes, I believed you. Shame on you for lying. No, I won't believe you next time, Shame on me if I'm conned again.
Was the 2024 election the last bona fide US election?
I'm not wanting to "gloom and doom" anyone. There have been several DU posts intimating that US democracy is over. There have been other posts that say, "let all the bad happen and then people will vote more reasonably."
The question is, "Will people be allowed to have a relatively free and fair election in the future?"
My caveat: I'm willing to concede there may have been election shenanigans. There may have been voter suppression, vote flipping, etc. But even so, 45 - 48% of actual voters voted for Trump. Even if Harris squeaked a win, the vote was too close that Trumpers would not have accepted a loss peacefully. If Trump is able to confirm his cabinet without any real Senate oversight, it's all over.
That is, I think the last election was fair "enough". Fair in that a large enough swath, in my opinion, the majority, voted for Trump that it was a fair election. If there are no checks and balances, I see no reason to expect that there will be another fair election. A good indicator will be the push-back R senators give to rubberstamping the incoming presidential cabinet.
I hope that everyone who has the energy and resources will continue to work for decency and the common good. Being either a very late Boomer or early Gen-Xer, my fight days are over. Physical ailments mean that I will ride out my time here on Earth. I'm not giving up. I'm concentrating on my own situation.
My CT thinking: What's to stop the Trump Admin from using AI to purge dissidents?
I hope this post sinks. I hope no one sees this as a possibility and this is dismissed as implausible.
Is there any country that has better positioned infrastructure to exploit a surveillance state than the US? I'm guessing the government can backdoor most of our electronic devices. My guess is the only thing currently stopping this is a just judicial system. If all judges become enforcers of authoritarianism, our information/communication system is ripe for exploitation.
AI currently hallucinates. I have read that this is due to the Transformer part of GPT (Generative Pre-trained Transformer). AI makes errors because it isn't "intelligent". It is modeled on neural networks. It is literally the mathematical dot product of an array of input against a matrix of numbers based on "trusted" data (the "P", or pre-trained in GPT) reiterated against itself until a final answer is determined. What AI is good at is finding unusual associations. By combining AI with standard procedural algorithms, it should be relatively easy to "thought police" a population.
The authoritarian state could use computers to tirelessly match up subversive posts from social media, anonymous message boards, reddit, well, everything. If a person is deemed undesirable, not only would it be easy to build a profile, but also, the state could plant any supportive evidence that it wanted on the accused's electronic devices.
AI could determine the rate at which citizenry would tolerate the re-education/compliance/termination of undesirables.
What if they told us exactly what they are going to do? "You know I was a snake. . ." -----"You know the story of the scorpion and the frog?" ---I don't remember any exact quotes, but the list is endless. He wants to kill his political enemies, to send the IRS after other people, the Judicial dept against others.
He is so vain of course he would get off on telling his supporters, "I'm coming after you next. . ." Musk promised we'd go through things. Trump basically told his supporters to save him from justice so he could "sting" them.
But the thing that worries me is in the title of this post. Can back doors to all of our electronic devices be used not only police thoughts, but also plant false evidence. If I were that nefarious and wanted to get rid of a political opponent, I would plant evidence of totally unrelated crimes or vices against that person.
Profile Information
Member since: Thu Sep 6, 2018, 01:26 AMNumber of posts: 403