AI

[JohnsonJones]

On 2/9/2023 at 12:41 AM, Vort said:

I don't know. I'm actually quite impressed by this. Take a look. I think very few students could do a better job.

 

This is something I've started to try to figure out.  I am worried about it.  There are different angles of attack various professors are taking to quell cheating.  It seems some students are using ChatGPT and other things to write their reports and essays these days. 

I don't know how to reliably detect what is or is not written by these AI's.  The AI detector ideas others have put out there seem to have their flaws.

The best I can think of is when they need to write an essay or something similar, to have them take it as a test in the testing center with a 6 hour window...

But that defeats the purpose of a lot of the reason to have the reports or essays written.  This is especially true when doing Historiography. 

Does anyone know of a reliable method on HOW to detect if a student is actually using one of these AI's to write their papers rather than the student writing it?

[NeuroTypical]

1 hour ago, JohnsonJones said:

Does anyone know of a reliable method on HOW to detect if a student is actually using one of these AI's to write their papers rather than the student writing it?

Yes.  You ask the student to summarize their paper, and take a sample of their work live. 

Here, let a millennial explain it:

 

[Vort]

For a mildly interesting read, ask ChatGPT to write a politically conservative SNL Weekend Update.

[Carborendum]

16 hours ago, NeuroTypical said:

You ask the student to summarize their paper, and take a sample of their work live. 

I wonder if that is a real test.  I can have AI write it.  Then I'd read it.  I could give a summary by reading the paper just once.  Others may have to read it a few times.  But it would take about the same amount of time.

I'd ask for more than a summary.  I'd ask them the what, why, when, where, & how.  How did they come up with a certain conclusion? 

Why did they take the position they did?  Did they originally have a different position, but became educated on the matter such that they had a change of mind?

You need to go deeper than a simple summary.

[NeuroTypical]

21 minutes ago, Carborendum said:

wonder if that is a real test.  I can have AI write it.  Then I'd read it.  I could give a summary by reading the paper just once.  Others may have to read it a few times.  But it would take about the same amount of time.

So, the two choices:

1. Learn the material and be able to show you understand it by writing about it and talking about what you write.

2. Use chatgpt and show you understand it by reading it until you can give a summary about it, and maybe include others in the process.

Honestly, #2 has some advantages. If I were a teacher, I'd have a class competition for "who can defend what they had chatgpt write the best?"   Students would get points for knowing what's actually in their papers.  Learning is still achieved. 

 

[Carborendum]

25 minutes ago, NeuroTypical said:

So, the two choices:

1. Learn the material and be able to show you understand it by writing about it and talking about what you write.

2. Use chatgpt and show you understand it by reading it until you can give a summary about it, and maybe include others in the process.

Honestly, #2 has some advantages. If I were a teacher, I'd have a class competition for "who can defend what they had chatgpt write the best?"   Students would get points for knowing what's actually in their papers.  Learning is still achieved. 

Do you remember the movie The Monkey's Uncle?

"It doesn't matter how you learn, so long as you learn."

I wonder at what point ChatGPT can just be considered a "study/research tool" vs "cheating"?  If used properly, could it be an "honest way to cheat"?

[NeuroTypical]

19 minutes ago, Carborendum said:

I wonder at what point ChatGPT can just be considered a "study/research tool" vs "cheating"? 

The answer is "every hour that goes by, another 10,000 humans join the first one".  I used the thing to help write my last sacrament meeting talk.  And to my astonishment, the part Chatgpt wrote was by far the crowd favorite.  I must have had 6-8 different people come up after to thank me for it.  Someone asked me to email it to them.  People were commenting on it 2 weeks later - totally unheard of for most sacrament meeting talks, especially mine.   I admitted what I had done to everybody, and reaction varied from intense interest, to awkward shared anxiety.  

I've also used it often to argue on the internet.  It's ability to source claims is far superior to any search engine.

Edited May 30, 2023 by NeuroTypical

[JohnsonJones]

2 hours ago, Carborendum said:

I wonder if that is a real test.  I can have AI write it.  Then I'd read it.  I could give a summary by reading the paper just once.  Others may have to read it a few times.  But it would take about the same amount of time.

I'd ask for more than a summary.  I'd ask them the what, why, when, where, & how.  How did they come up with a certain conclusion? 

Why did they take the position they did?  Did they originally have a different position, but became educated on the matter such that they had a change of mind?

You need to go deeper than a simple summary.

 

This could work, but I have over 500 students in the undergrad classes (edit for clarity: For the department overall) and limited time.  The reason they write papers in the first place was to have them answer these types of questions, getting them to do the studying on their own, without taking up more class time to go over it.  Office hours would be for those who had problems with these things to get more in depth, but normally I don't have most of the students going to these (or even one or two during the beginning of the semester).

Graduate students are not what I'm worried about currently (we work much more closely with them, and generally are familiar with their writing style.  If that changes drastically, that could raise questions though). 

I've seen stories of other professors trying to get students using these ChatGPT things and stories where they were mistaken and accused the students falsely.  I don't want to be that type of paranoid professor.  At the same time, students who use this are cheating themselves as well.  Eventually if they wish to go into graduate studies they will need to write a thesis and papers with more overview by a professor.   If they do not gain the skills when they are undergraduates, it could really hurt them later.

Of course, that's assuming this entire AI thing doesn't revolutionize the entire discourse.  It could be that this entire AI thing eventually becomes the TOOL to use rather than our own ways of doing it presently (Sort of how the computer replaced the typewriter).  I can't predict the future, but it seems a little bothersome to me currently.

I can't tell the difference, if they are actually using it or not (possible no students are, but with the prevalence of stories I have heard coming out, I would be surprised if there was at least a few using it.  

I wonder how they figure out which footnotes to attach to it though, and how to verify those footnotes actually say what they do? 

Edited May 30, 2023 by JohnsonJones

[Carborendum]

4 hours ago, NeuroTypical said:

The answer is "every hour that goes by, another 10,000 humans join the first one". 

I don't quite follow what your meaning is here.  Are you talking about the approximately 14,000 people born every hour?

I'm not sure how that answers my question about research tool vs cheating.

4 hours ago, NeuroTypical said:

I used the thing to help write my last sacrament meeting talk.  And to my astonishment, the part Chatgpt wrote was by far the crowd favorite.  I must have had 6-8 different people come up after to thank me for it.  Someone asked me to email it to them.  People were commenting on it 2 weeks later - totally unheard of for most sacrament meeting talks, especially mine.   I admitted what I had done to everybody, and reaction varied from intense interest, to awkward shared anxiety.  

I've also used it often to argue on the internet.  It's ability to source claims is far superior to any search engine.

I'm going to echo someone else's previous post that you share this ChatGPT generated talk.  I'd be interested in what made it so popular.

[NeuroTypical]

4 hours ago, NeuroTypical said:

5 hours ago, Carborendum said:

I wonder at what point ChatGPT can just be considered a "study/research tool" vs "cheating"? 

The answer is "every hour that goes by, another 10,000 humans join the first one". 

What I mean:  AI tech is very new, and the whole human race is trying to figure out what it means, and what to do with it.   As time goes on, more and more people think of it as a study/research tool.  In my opinion, it's only a matter of time before AI is integrated into our educational methodologies.  The same way computers were.  The same way calculators were.

37 minutes ago, Carborendum said:

share this ChatGPT generated talk.

Well, I wrote the talk, and asked chatgpt to write a poem.  I'll share the prompt:

Write a poem about how a mormon can tell if they are worthy to partake of the sacrament.  Specifically mention common issues including: being convicted of serious crime, abortion, murder, sexual misconduct, polygamy, surgical transition, felony, law of chastity, donated your eggs, fertility treatments, in vitro fertilization, victim of abuse, inadvertent exposure to pornography, prejudiced, same sex attraction, single parent, unwed and pregnant, surrogate mother, chosen cremation, HIV or AIDS, ambiguous genitalia, medical MJ, organ donation, lack of vaccination, shopping on Sunday, word of wisdom struggles, a smoker, a drinker, an addict, a slave, a gambler, illegal immigrant.

I removed a bunch of the results, tinkered with maybe 15% of it, and then read the thing at the end of my talk.  Feel free to stick the prompt into chatgpt to see what it gives you.

[mordorbund]

On 5/29/2023 at 11:59 AM, Carborendum said:

This guy learned the hard way that trusting & depending on AI can get you into trouble.

https://www.bbc.com/news/world-us-canada-65735769

SUMMARY:

A lawyer used ChatGPT to look up case law.  It turned out that there were a number of bogus cases that the AI cited.  The lawyer didn't bother to look up the standard sources to verify the cases and what their contents were.

When the opposing legal team looked for the cases, they either couldn't find the cases at all, or they found numerous errors in the citations.

What's worse is that the guy doing the research wasn't even the lead counsel.  So, the lead counsel was reprimanded for this other guy's failure to verify the sources.

 

I don't know why the lawyer would do that.  If you just want help looking things up, sure, use AI to help you find the relevant  cases.  But then verify them through manual means.

When I'm looking up various codes, I will use search engines and other tools to determine what passage is even talking about my situation.  But when I need to cite them, I have to find the original source and get the specific wording from the original source.  Then I make sure it has the right context and applicability to my situation.

Won’t be long before ChatGPT starts editing Wikipedia for us.

[mordorbund]

I’ve got a friend who’s an Econ professor. He had ChatGPT write an essay in response to one of his standard essay questions. He then had the students correct the response as an assignment. That seems like a fair way of acknowledging that students will want to use the AI while letting them know in practical terms that they’ll still have to put in some work.

@JohnsonJones you’ve been in the business long enough to remember when Wikipedia came on the scene. There’s some comparability here.

[mordorbund]

5 minutes ago, mordorbund said:

Won’t be long before ChatGPT starts editing Wikipedia for us.

The post just reminded me of the Amelia Bedilia author’s entry, which had some made up info, which was referenced but not cited by a major paper, which was then used as a citation to bolster the made up claim.

We can streamline the process by having Chat edit wikis with its false citations, get published and then re-edit the wiki with the newly-published article as a source.

[Carborendum]

On 5/30/2023 at 6:50 PM, NeuroTypical said:

Feel free to stick the prompt into chatgpt to see what it gives you.

Unfortunately, I take my privacy pretty seriously.  And it required me to give up some private information to use it.  I have no account.

[NeuroTypical]

I totally get it.  I have a dummy/junk/spam account for such purposes.  But yeah, respect to you!

[Guest Godless]

Whelp, humanity had a good run.

 

[askandanswer]

The teaching of math survived the introduction of calculators. Perhaps that experience might shed some light on how education can make good use of chatgpt. 

ANd @JohnsonJonesperhaps you should ask chatgpt how you can detect essays it has written.

I've marked a few essays lately and I think I'm starting to recognise its writing style. My guess is that the final paragraph of an essay written by chatgpt is far more likely to start with the words "In conclusion" than a final paragraph written by a student. Another possibility is for you to ask chatgpt to write an essay on the topic you have assigned the students to write about and then look to see how many essays come in that closely resemble the one you asked chatgpt to write. 

[Carborendum]

8 hours ago, askandanswer said:

The teaching of math survived the introduction of calculators. Perhaps that experience might shed some light on how education can make good use of chatgpt. 

I would disagree.

While I do accept that calculators are a valuable tool, just as computers are (basically glorified calculators) there is a problem with using calculators too early in the process.  If you use calculators at the very beginning of teaching basic arithmetic, the student will be hampered in learning head math.

In engineering school, I was taught manual methods of calculating things so that I could understand how the physics of things actually worked.  Then when I used software to do the actual analysis, I noticed when something was off.  I also had some idea as to why it was off.  Then I could make proper adjustments to either the input or the design to provide an adequate fix.

Today, students are taught 100 and 200 level analysis by hand.  But the 300 level and above, they are taught to mostly use software.  And the kids coming out of engineering schools today are just not cutting the mustard.

[Traveler]

10 hours ago, askandanswer said:

The teaching of math survived the introduction of calculators. Perhaps that experience might shed some light on how education can make good use of chatgpt. 

This is an interesting paradigm.   I will speak to my experience with education and adjustments I took in relationship to spelling.  I am dyslexic.  This is not a condition that one is or is not so much as it is a spectrum.  My place in the spectrum caused great difficulty with spelling.  I have never been able to pass a spelling test in my life.  And it has strange outcomes for me.  Sometimes I get confused over simple 3 letter words and sometimes very complex and weird spelt words come out correct.  Spell check is a godsend for me.

Somewhere in grade school I learned to read by taking in lines of words at a time rather than single words.  If I am reading out loud, I will often repeat what I read out of order or with different words.  This seems to upset some people.  I have a brother that can read a chapter in a book and repeat it back word for word.  I read a chapter and can repeat it back, but it is in summary form and often in a different order.  I have found that if I read a chapter backwards, I can get the jest of things and can skip portions. 

My point is that many have what experts think are learning difficulties.  I think many such difficulties are because of methods.  We think because we learn based in a certain method that everybody likewise needs the same method to succeed.  I have tried to teach others of my methods and it is only worse for them.

If AI becomes capable of realizing the difference in individual learning abilities – education and the success of students will change dramatically.  I am concerned that AI will be a reflection of algorithms that we create based in our design of what we think is intelligence.  And like so many advances – a benefit for some and the opposite for others.  But there is a chance that AI will recognize patterns missed by most educators.  I am not so fearful of AI as others.  I believe we are on a threshold of many things.  I do wonder what AI would do with LGBTQ+ intrusions into society?  I am quite sure that if the logic and algorithms of mathematical fractals are at the heart of AI implementation – certain segments of our population will be very unhappy with the implementation of AI.

 

The Traveler

[NeuroTypical]

By the way, it totally could have gone down this way:

 

[Carborendum]

40 minutes ago, NeuroTypical said:

I'm sorry. 

I got stuck thinking:  How does Vader eat without removing that mask?

[Vort]

1 hour ago, NeuroTypical said:

By the way, it totally could have gone down this way:

That was more than merely funny. In a counterintuitive and perhaps stupid way, I found it touching. Darth Vader in white, regularly attending the temple. It's sort of like Satan reclaiming his ancient title of Lucifer; absurd, perhaps even offensive, but deep down it's a nice thought.

[Vort]

42 minutes ago, Carborendum said:

How does Vader eat without removing that mask?

Ask Spiderman.

[Carborendum]

2 hours ago, Vort said:

Ask Spiderman.

We see that Spiderman removes his mask.  But (despite the scene in ESB)  Luke said that if Vader removed the mask, he'd die.  And he didn't have his "chamber" available at the pot luck.

Edited June 10, 2023 by Carborendum

[Ironhold]

5 hours ago, Vort said:

That was more than merely funny. In a counterintuitive and perhaps stupid way, I found it touching. Darth Vader in white, regularly attending the temple. It's sort of like Satan reclaiming his ancient title of Lucifer; absurd, perhaps even offensive, but deep down it's a nice thought.

It's not the first time Vader has been depicted in white. 

There was an alternate continuity comic series, sort of a "what-if?" type of thing, in which Luke actually gets through to Vader during Empire. This causes Vader to don a white outfit and offer his services in hunting down Palpatine to help atone for his actions.