Discussion and links topic.
It would be nice if this could lead to widespread skepticism and interest in critical thinking. In the video one of the interviewees talks about an aspect of educational content is that people are more trusting of the content and feel they can walk away being informed, but now people are potentially walking away being less informed because of low quality AI stuff. I think everyone could learn to be more skeptical/critical in general of educational resources, AI or not. Some people might become despondent over the situation of so much slop though and give up.
I think lots of people care whether a piece of media is AI content, and prefer to not consume it, and an issue is identifying it easily. Recently, Spotify has begun labelling AI-generated music. I think that’s helpful.
I just got to the part of the video where it talks about the AI watermarking and stuff that YouTube and the AI companies are doing to make identifying it easy. I think that’s all good stuff.
Two recent stories about mathematicians using LLMs to assist with their work for months, then OpenAI using LLMs to solve what they were working on and publishing first, potentially by using their drafts and other work as training data:
If nonpublic research supplied by users improved a model and the provider then used that model to race those users to publication—without informed consent, disclosure, or credit—that would be ethically indefensible. De-identification may remove a name; it does not remove the intellectual content of a mathematical idea.
I asked when the first prompt had been sent by them. This question was not answered directly by OpenAI for some time. Eventually it was agreed that it had been sent in the past few days, after information about our work had reached OpenAI.
I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer.
and
I said that if OpenAI released its result in the way proposed I would go public with what happened. The reply was, “Why would you ruin your career?” I replied that I am an academic, and asked why he thought going public would ruin my career. The reply was, “If you don’t want me to be nice, then I don’t have to be nice.”
Something similar can now happen with CF and a ton of other ideas too. LLMs have trained on public CF essays. A user could talk to an LLM about philosophy and reinvent a lot of CF, with the LLM being very helpful and supplying a lot of the ideas, without the person knowing they were plagiarizing. CF is already published but isn’t well known enough to stop someone else from potentially getting credit for lots of it. Also, I do have draft additions to CF that I’ve worked on and talked with LLMs about.