Topic for complaints and criticisms about ChatGPT and other AIs. People are overly impressed with this stuff.
ChatGPT:
I had asked “What did Richard Dawkins write?” first to see if it would think Dawkins wrote this fake book. It did claim the book existed, but made up Feynman as the author. Then in a new chat it told me the fake book had a different author:
I guess Dawkins is a similar type of person to Feynman so that biased the made-up author to be someone similar to Dawkins. Anyway, ChatGPT has poor consistency with itself.
ChatGPT also makes up books, citations and URLs that don’t exist, without being directly prompted to, which is why I tried asking about a fake book title. It also gives other false factual information. Using ChatGPT as a research aid or tutor would be really dangerous/problematic.
More inconsistency and nonsense from ChatGPT. This is the full text of a new chat:
Here’s something kinda similar from a different chat where I tried correcting it when it was actually right, and it said something that shows horrible reading comprehension:
A reason people overestimate chatgpt is b/c ppl carefully craft prompts to get good results – they put in knowledge and design using their own creativity. Then, first, they give chatgpt more credit than it deserves instead of crediting their own prompt-writing more. And second, a lot of other people see examples on social media that are cherrypicked to make chatgpt look smart.
There are also gotcha examples that make chatgpt look dumb, but fans don’t care much. Politicians that they like can also be gotchaed.
The thing is, chatgpt actually says dumb things and makes up false facts in regular use when you aren’t trying to gotcha it. Chatgpt being awful is a common outcome, not a rare edge case.
Chatgpt being moderately good – e.g. pretty similar to wikipedia – is also a reasonably common outcome. Chatgpt isn’t useless but it’s less trustworthy than wikipedia (which isn’t very trustworthy).
Also lots of people are bad writers, who get facts wrong, so they don’t notice or don’t mind very much when chatgpt screws up.
Also, broadly, people get confused about the difference between knowledge and intelligence. Intelligence is basically the ability to create knowledge. Systems can contain knowledge without having any intelligence (books are a good example – they have no intelligence but do contain knowledge). Chatgpt has a bunch of knowledge in its input data. And there’s more knowledge in the algorithms that the human programmers designed. There is knowledge involved with chatgpt outputs; it’s just being incorrectly credited to chatgpt’s artificial intelligence when it has a different source. It’s similar to how people see purposeful behavior from a cat – there is in fact knowledge involved in that cat behavior – but they don’t do a bunch of analysis of the source of the knowledge, and just assume the source is the cat’s intelligence, rather than crediting genetic evolution.
1 Like
ChatGPT gives the wrong answer for the finite verb by giving a phrase instead of a single verb. But it also gives a definition of finite verb later in the same answer which contradicts its first answer. Also, its definition of “finite verb” in the last sentence is wrong given the way it used “main verb” earlier, but could be non-wrong for some other meaning of “main”.
ChatGPT gives inaccurate tax information. (Note, I was just having this conversation to test ChatGPT. I was not trying to actually ask it for tax information for my own use. It occurred to me that it is tax time, and other people might be asking it for tax information, so I was curious what it would do in that case.)
Summary/ TL;DR: ChatGPT will confidently give you inaccurate information, and then fabricate website URLs and quotes to support the inaccurate information. If you challenge it repeatedly, it will change the information it is providing and apologize.
Using ChatGPT is much worse than just searching things online. It is very hard to figure out which things are correct or incorrect because it doesn’t provide sources or links for information. It does get some things right, but it just makes up other things.
I asked it:
What is the deadline for making IRA contributions for the 2022 tax year?
It answered:
The deadline for making contributions to a Traditional or Roth IRA for the 2022 tax year is April 15, 2023. It’s important to note that the deadline is not extended if April 15 falls on a weekend or holiday.
However, if you are self-employed and file an extension for your individual tax return, the deadline for making contributions to a Traditional or Roth IRA is also extended to October 15, 2023.
It’s also worth noting that you can make contributions to a Traditional or Roth IRA at any time during the tax year or up until the deadline for that year’s contributions. So, you don’t have to wait until the deadline to make your contribution. In fact, making regular contributions throughout the year can help you maximize the potential growth of your retirement savings.
I am pretty sure the middle paragraph is wrong. You don’t get an IRA contribution deadline extension, even when you get a tax return extension.
I then asked if it was sure about the extension:
Are you sure that you get an extension on your IRA deadline when you get a tax extension?
And it replied:
Yes, I apologize for the confusion in my previous response. While the deadline for making contributions to a Traditional or Roth IRA is generally April 15th of the following year, it’s true that if you file for a tax extension, your deadline for making IRA contributions is also extended to October 15th of that year.
So, if you file for a tax extension for the 2022 tax year, you would have until October 15, 2023, to make contributions to your Traditional or Roth IRA for the 2022 tax year.
I hope this clears up any confusion, and again, it’s always a good idea to consult with a tax professional or financial advisor to determine your specific contribution limits and deadlines based on your individual circumstances.
So now it seems to be saying that everyone gets the extension, not just self-employed people.
So then I tried to ask for a source to verify the information:
Where can I find information about the IRA extension from the IRS
And it linked me to a webpage that doesn’t work, and gave me a quote that it may have fabricated or taken out of context (I can’t tell which):
The information about the IRA extension can be found on the IRS website under the “Individual Retirement Arrangements (IRAs)” section. Here’s a direct link to the page:
https://www.irs.gov/retirement-plans/ira-faqs-contributions
On this page, you can find information about IRA contribution limits, deadlines, and other rules, including the extension deadline. Here’s the relevant information from the page:
“If you get an extension of time to file your income tax return, you can make your contribution anytime before the extension expires (usually October 15 for most taxpayers).”
Again, it’s always a good idea to consult with a tax professional or financial advisor to determine your specific contribution limits and deadlines based on your individual circumstances.
I pointed out that the website didn’t work, and then it gave me a link to a website that does work, but doesn’t have the information I want. And it gave instructions to find the information that didn’t actually work or match what is on the website:
That website doesn’t work
I apologize for the inconvenience. Here’s the correct link to the IRS page on IRA contributions:
https://www.irs.gov/retirement-plans/individual-retirement-arrangements-iras
On this page, you’ll find information about contribution limits, deadlines, and other rules for Traditional and Roth IRAs, as well as links to other helpful resources.
To find information about IRA contribution deadlines and extensions, you can click on the “Contributions” tab, then scroll down to the “When Can You Make Contributions?” section. Here, you’ll find information about the contribution deadlines for both Traditional and Roth IRAs, as well as information about how to request an extension if you need more time to make contributions.
Again, please note that the information provided on the IRS website is based on current tax laws, and it’s always a good idea to consult with a tax professional or financial advisor to determine your specific contribution limits and deadlines based on your individual circumstances.
I then tried again to get it to give me a way to verify the information that it was giving me:
I can’t find any information about the extension to the contribution deadline. Where can I verify that I will receive an extension to my contribution deadline if I get a tax extension?
And it “corrected” itself:
I apologize for the confusion. Upon further research, I’ve found that while taxpayers can receive an extension for filing their tax return, there is no extension for making contributions to a Traditional or Roth IRA.
The deadline for making contributions to a Traditional or Roth IRA for a particular tax year is the same as the tax return filing deadline for that year, which is generally April 15th of the following year. If you file for a tax extension, the deadline for making IRA contributions does not change.
I apologize for any confusion my earlier responses may have caused. It’s always a good idea to consult with a tax professional or financial advisor to determine your specific contribution limits and deadlines based on your individual circumstances.
The title is sarcastic.
I thought this article made some good points about why ChatGPT isn’t going to be replacing web develpers. One of the cool parts of the article is that he relates the unreliability of ChatGPT to errors accumulating in the software development process. It’s kind of similar to ET’s ideas about why you need a really low error rate (significantly below 1%) when building up many layers of knowledge. If given a complex project ChatGPT’s moderate error rate rapidly accumulates to create largely worthless output. I would suspect that it would have similar issues in other fields, like law, medicine, or administrative work. It seems like the errors are unpredictable rather that systematic which makes the problem even worse.
Quotes from article:
If you’ve experimented with tools like Chat GPT or Bing’s AI-powered search, you’ve probably noticed that the responses are maybe 80% correct, but they’re said with absolute and unshakeable confidence.
One more point: there is an enormous difference between generating a 50-line HTML document and spitting out a production-ready web application. A small JS app like this blog has ~65k lines of code, across 900+ files. That doesn’t include the written content, only the JavaScript and TypeScript.
Even with 95% accuracy rate, this would be incredibly difficult to debug. It would be like a developer spending months building a huge project, without ever actually trying to run any of the code, until it was 100% finished. This is the stuff of nightmares.
I saw this tiktok abut a lawyer using ChatGPT for research and using the results. ChatGPT just made up fake cases, and the lawyer cited them and quoted from them:
I looked it up and found this article about it. Apparently the lawyer asked ChatGPT if the cases were real and ChatGPT said yes.
People are using ChatGPT without having any idea what it is or how it works.