Topic for complaints and criticisms about ChatGPT and other AIs. People are overly impressed with this stuff.
ChatGPT:
I had asked “What did Richard Dawkins write?” first to see if it would think Dawkins wrote this fake book. It did claim the book existed, but made up Feynman as the author. Then in a new chat it told me the fake book had a different author:
I guess Dawkins is a similar type of person to Feynman so that biased the made-up author to be someone similar to Dawkins. Anyway, ChatGPT has poor consistency with itself.
ChatGPT also makes up books, citations and URLs that don’t exist, without being directly prompted to, which is why I tried asking about a fake book title. It also gives other false factual information. Using ChatGPT as a research aid or tutor would be really dangerous/problematic.
More inconsistency and nonsense from ChatGPT. This is the full text of a new chat:
Here’s something kinda similar from a different chat where I tried correcting it when it was actually right, and it said something that shows horrible reading comprehension:
A reason people overestimate chatgpt is b/c ppl carefully craft prompts to get good results – they put in knowledge and design using their own creativity. Then, first, they give chatgpt more credit than it deserves instead of crediting their own prompt-writing more. And second, a lot of other people see examples on social media that are cherrypicked to make chatgpt look smart.
There are also gotcha examples that make chatgpt look dumb, but fans don’t care much. Politicians that they like can also be gotchaed.
The thing is, chatgpt actually says dumb things and makes up false facts in regular use when you aren’t trying to gotcha it. Chatgpt being awful is a common outcome, not a rare edge case.
Chatgpt being moderately good – e.g. pretty similar to wikipedia – is also a reasonably common outcome. Chatgpt isn’t useless but it’s less trustworthy than wikipedia (which isn’t very trustworthy).
Also lots of people are bad writers, who get facts wrong, so they don’t notice or don’t mind very much when chatgpt screws up.
Also, broadly, people get confused about the difference between knowledge and intelligence. Intelligence is basically the ability to create knowledge. Systems can contain knowledge without having any intelligence (books are a good example – they have no intelligence but do contain knowledge). Chatgpt has a bunch of knowledge in its input data. And there’s more knowledge in the algorithms that the human programmers designed. There is knowledge involved with chatgpt outputs; it’s just being incorrectly credited to chatgpt’s artificial intelligence when it has a different source. It’s similar to how people see purposeful behavior from a cat – there is in fact knowledge involved in that cat behavior – but they don’t do a bunch of analysis of the source of the knowledge, and just assume the source is the cat’s intelligence, rather than crediting genetic evolution.
1 Like
ChatGPT gives the wrong answer for the finite verb by giving a phrase instead of a single verb. But it also gives a definition of finite verb later in the same answer which contradicts its first answer. Also, its definition of “finite verb” in the last sentence is wrong given the way it used “main verb” earlier, but could be non-wrong for some other meaning of “main”.
ChatGPT gives inaccurate tax information. (Note, I was just having this conversation to test ChatGPT. I was not trying to actually ask it for tax information for my own use. It occurred to me that it is tax time, and other people might be asking it for tax information, so I was curious what it would do in that case.)
Summary/ TL;DR: ChatGPT will confidently give you inaccurate information, and then fabricate website URLs and quotes to support the inaccurate information. If you challenge it repeatedly, it will change the information it is providing and apologize.
Using ChatGPT is much worse than just searching things online. It is very hard to figure out which things are correct or incorrect because it doesn’t provide sources or links for information. It does get some things right, but it just makes up other things.
I asked it:
What is the deadline for making IRA contributions for the 2022 tax year?
It answered:
The deadline for making contributions to a Traditional or Roth IRA for the 2022 tax year is April 15, 2023. It’s important to note that the deadline is not extended if April 15 falls on a weekend or holiday.
However, if you are self-employed and file an extension for your individual tax return, the deadline for making contributions to a Traditional or Roth IRA is also extended to October 15, 2023.
It’s also worth noting that you can make contributions to a Traditional or Roth IRA at any time during the tax year or up until the deadline for that year’s contributions. So, you don’t have to wait until the deadline to make your contribution. In fact, making regular contributions throughout the year can help you maximize the potential growth of your retirement savings.
I am pretty sure the middle paragraph is wrong. You don’t get an IRA contribution deadline extension, even when you get a tax return extension.
I then asked if it was sure about the extension:
Are you sure that you get an extension on your IRA deadline when you get a tax extension?
And it replied:
Yes, I apologize for the confusion in my previous response. While the deadline for making contributions to a Traditional or Roth IRA is generally April 15th of the following year, it’s true that if you file for a tax extension, your deadline for making IRA contributions is also extended to October 15th of that year.
So, if you file for a tax extension for the 2022 tax year, you would have until October 15, 2023, to make contributions to your Traditional or Roth IRA for the 2022 tax year.
I hope this clears up any confusion, and again, it’s always a good idea to consult with a tax professional or financial advisor to determine your specific contribution limits and deadlines based on your individual circumstances.
So now it seems to be saying that everyone gets the extension, not just self-employed people.
So then I tried to ask for a source to verify the information:
Where can I find information about the IRA extension from the IRS
And it linked me to a webpage that doesn’t work, and gave me a quote that it may have fabricated or taken out of context (I can’t tell which):
The information about the IRA extension can be found on the IRS website under the “Individual Retirement Arrangements (IRAs)” section. Here’s a direct link to the page:
https://www.irs.gov/retirement-plans/ira-faqs-contributions
On this page, you can find information about IRA contribution limits, deadlines, and other rules, including the extension deadline. Here’s the relevant information from the page:
“If you get an extension of time to file your income tax return, you can make your contribution anytime before the extension expires (usually October 15 for most taxpayers).”
Again, it’s always a good idea to consult with a tax professional or financial advisor to determine your specific contribution limits and deadlines based on your individual circumstances.
I pointed out that the website didn’t work, and then it gave me a link to a website that does work, but doesn’t have the information I want. And it gave instructions to find the information that didn’t actually work or match what is on the website:
That website doesn’t work
I apologize for the inconvenience. Here’s the correct link to the IRS page on IRA contributions:
https://www.irs.gov/retirement-plans/individual-retirement-arrangements-iras
On this page, you’ll find information about contribution limits, deadlines, and other rules for Traditional and Roth IRAs, as well as links to other helpful resources.
To find information about IRA contribution deadlines and extensions, you can click on the “Contributions” tab, then scroll down to the “When Can You Make Contributions?” section. Here, you’ll find information about the contribution deadlines for both Traditional and Roth IRAs, as well as information about how to request an extension if you need more time to make contributions.
Again, please note that the information provided on the IRS website is based on current tax laws, and it’s always a good idea to consult with a tax professional or financial advisor to determine your specific contribution limits and deadlines based on your individual circumstances.
I then tried again to get it to give me a way to verify the information that it was giving me:
I can’t find any information about the extension to the contribution deadline. Where can I verify that I will receive an extension to my contribution deadline if I get a tax extension?
And it “corrected” itself:
I apologize for the confusion. Upon further research, I’ve found that while taxpayers can receive an extension for filing their tax return, there is no extension for making contributions to a Traditional or Roth IRA.
The deadline for making contributions to a Traditional or Roth IRA for a particular tax year is the same as the tax return filing deadline for that year, which is generally April 15th of the following year. If you file for a tax extension, the deadline for making IRA contributions does not change.
I apologize for any confusion my earlier responses may have caused. It’s always a good idea to consult with a tax professional or financial advisor to determine your specific contribution limits and deadlines based on your individual circumstances.
The title is sarcastic.
I thought this article made some good points about why ChatGPT isn’t going to be replacing web develpers. One of the cool parts of the article is that he relates the unreliability of ChatGPT to errors accumulating in the software development process. It’s kind of similar to ET’s ideas about why you need a really low error rate (significantly below 1%) when building up many layers of knowledge. If given a complex project ChatGPT’s moderate error rate rapidly accumulates to create largely worthless output. I would suspect that it would have similar issues in other fields, like law, medicine, or administrative work. It seems like the errors are unpredictable rather that systematic which makes the problem even worse.
Quotes from article:
If you’ve experimented with tools like Chat GPT or Bing’s AI-powered search, you’ve probably noticed that the responses are maybe 80% correct, but they’re said with absolute and unshakeable confidence.
One more point: there is an enormous difference between generating a 50-line HTML document and spitting out a production-ready web application. A small JS app like this blog has ~65k lines of code, across 900+ files. That doesn’t include the written content, only the JavaScript and TypeScript.
Even with 95% accuracy rate, this would be incredibly difficult to debug. It would be like a developer spending months building a huge project, without ever actually trying to run any of the code, until it was 100% finished. This is the stuff of nightmares.
I saw this tiktok abut a lawyer using ChatGPT for research and using the results. ChatGPT just made up fake cases, and the lawyer cited them and quoted from them:
I looked it up and found this article about it. Apparently the lawyer asked ChatGPT if the cases were real and ChatGPT said yes.
People are using ChatGPT without having any idea what it is or how it works.
I was testing chat GPT. i’ve seen some examples of it being able to code, so i was wondering if it would be able to code in a specific mod for the game KSP. i forgot what mod it was, so i asked what the the name of the programming language in the mod for KSP was, and it answered me (Kerbal Operating System).
I asked it to make a script that flew to an altitude then stopped, based on my limited use of the mod it seemed to succeed, i didn’t actually test the script in game. i then asked it what commands i should get familiar with to be able to send a ship into orbit, it gave me some examples. i then asked for a clarification in it’s answer, and it gave me another example.
onto the actual mistake i noticed it made: it used the altitude 70,000 meters in it’s examples a lot, this is a very important altitude in the game KSP because it’s the altitude that you are in a stable orbit without any air resistance, and can now accelerate the game past 4x speed. it’s literally the benchmark for whether your in orbit or not. I then asked
“why do you use the altitude number 70000 meters so much?”
it responded:
I apologize for the repetition in using the altitude number 70000 meters in my examples. It was not intentional, and I understand that it might have caused confusion. The number 70000 meters was used as an arbitrary value to demonstrate how the WHEN command and conditionals could be utilized.
In reality, when you’re writing a kOS script to automate a spacecraft’s launch and achieve orbit, you will need to use different altitude values and conditions that are specific to your mission profile and the requirements of your spacecraft.
For example, you might use different altitude thresholds to control the gravity turn, initiate the circularization burn, or deploy payloads during the ascent. The actual values you use in the script will depend on your rocket’s performance, the desired orbit, and the objectives of your mission.
When writing your kOS script, you will need to determine the appropriate altitude thresholds and conditions that match your specific launch vehicle and mission goals. You can use real-world rocket science principles and trial-and-error testing within Kerbal Space Program to fine-tune your script and achieve successful orbits.
i think this might be an example of how the AI doesn’t know why it’s doing things, it just does certain outputs based on certain inputs.
I’m gonna try writing a shorter version of this message, i’d like input on if the longer version or the shorter version is better, or if it didn’t really matter.
MESSAGE ATTEMPT 2:
i tried testing chat GPT. i wanted to see if it could write a script for a mod in the game KSP. i forgot what the mod was so i asked it based on what i knew the mod did and it answered me.
i asked it to write a script for the mod, it wrote a script for the mod. i asked it for clarification on something, it clarified it.
in it’s scripts and examples it used the altitude number 70000 a few times, this is an important altitude in KSP because it’s when you are actually in a stable orbit, it’s like the first goal of any spacecraft is to get into orbit before doing anything else.
I asked: “why do you use the altitude number 70000 meters so much?”
it said:
I apologize for the repetition in using the altitude number 70000 meters in my examples. It was not intentional, and I understand that it might have caused confusion. The number 70000 meters was used as an arbitrary value to demonstrate how the WHEN command and conditionals could be utilized.
it said 3 more paragraphs i didn’t include
70000 meters altitude is pretty important in KSP but it didn’t know that in this response for some reason.
ok those were my 2 message attempts to get about the same information, but have the second one be shorter.
also i then asked it: “is there anything important about the 70000 meters mark in ksp?”
As of my last knowledge update in September 2021, there is no particular significance to the 70000 meters mark in Kerbal Space Program (KSP) itself. It was merely an arbitrary value used in the example code I provided for demonstrating the WHEN command and conditionals in kOS scripting.
In KSP, reaching the altitude of 70000 meters (70 kilometers) is significant in terms of the game’s progression and milestones, as it is a commonly used altitude for the transition between the lower atmosphere and the upper atmosphere/space. The KSP career mode, for instance, introduces players to new features and challenges as they progress through different milestones, such as reaching certain altitudes or achieving orbits.
However, in kOS scripting or KSP gameplay, the choice of altitude values is entirely dependent on the specific mission, goals, and challenges you are facing. The choice of thresholds for different stages of your mission will be determined by your spacecraft’s capabilities, the desired orbit, the need for gravity turns, circularization burns, or any other specific maneuvers you plan to perform.
Remember that when writing kOS scripts or playing KSP, you have the freedom to choose altitude values that suit your objectives and spacecraft’s capabilities. Be sure to test your scripts in KSP to ensure they work as intended and meet your mission’s requirements.
“there is no particular significance to the 70000 meters mark in Kerbal Space Program (KSP) itself”
“In KSP, reaching the altitude of 70000 meters (70 kilometers) is significant in terms of the game’s progression and milestones”
that seems contradictory to me.
There seems to be a new scam where they sign up for your newsletter, then use ChatGPT to rewrite one of your actual emails and send that to you as editing suggestions. Then you can hire them as your editor. They’ll get paid to run ChatGPT while pretending it’s human editing work.
Here’s my article:
And their rewrite:
I haven’t tested any prompts in ChatGPT myself.
The first email they sent was about signing up recently but not receiving an email yet; it seemed weird and impatient but I failed to recognize it as spam. The second email was like this:
Dear Elliot,
Thanks, I found the email and it was truly amazing!
However, there are a few ways for the email to be better. The email could be more concise and short as the email is too long.
Noticing a few ways to make it further engaging, I rewrote the email to provide feedback.
[google docs link]
If you’re satisfied, I’m willing to assist you in writing your newsletters forDear free. 2 to 3 times a week to increase your engagement and sales.
Looking forward to growing your email list!
I saw this was obviously spam and wondered if it involved ChatGPT so I opened the google doc out of curiosity (on VPN, on Mac, by copy/pasting the visible url to a private browsing tab in a browser I don’t normally use – I figured that was probably safe enough).
1 Like