I experimented with ChatGPT for creating introductory plagiarism-free content for Cybersecurity, and here are my results

After experimenting with this AI-powered chatbot based on GPT-3 for two weeks, I found this can’t be used for work which requires zero or low plagiarism

Image by Gerd Altmann from Pixabay

One of the things I’m constantly working on is explaining technical concepts in a way a 5th grader can understand. I have a few initiatives and discussions in progress where I’m trying to do precisely the same, explain a complex subject in the simplest way to 10–15 years old students.

When it comes to Cybersecurity, there is a lot of content on the Internet. However, getting lost in the vast sea of unlimited details, explanations, and complexity is easy.

I have been following OpenAI for almost three years now and have been various examples in my guest lectures and talks. When they introduced ChatGPT, it was clear that this would change how content is created. Not everyone needs to create literary and highly researched journal-grade content. When I tried this, I found that my intuition was correct.

Since its release, people have been using ChatGPT for various purposes — creating content, solving problems, and even generating program source code.

My experiments with creating a plagiarism-free content

So, I tried to ask ChatGPT about the first thing which comes to mind when getting into the area of Cybersecurity, the term “Cybersecurity” itself.

I asked ChatGPT, “Explain cybersecurity in simple terms”.

Here’s the answer I got:

Cybersecurity refers to the practices and technologies that are used to protect computers, networks, and devices from digital attacks, theft, and damage. These attacks can come in the form of viruses, malware, phishing scams, or other types of malicious software that can steal or damage data or disrupt the normal functioning of a system.

To protect against these threats, organizations and individuals use a variety of measures, such as installing antivirus software, creating strong passwords, and implementing secure networks and protocols. They may also use firewalls, intrusion detection systems, and other technologies to prevent unauthorized access to their systems.

In addition to these technical measures, good cybersecurity practices also include educating employees and users about how to identify and avoid potential threats, as well as having policies and procedures in place to respond to and recover from cyber-attacks.

Overall, the goal of Cybersecurity is to ensure the confidentiality, integrity, and availability of information and systems, and to protect against unauthorized access or attacks that could compromise these assets.

As a Cybersecurity professional, I can say this was an excellent answer covering most aspects.

This is where my experiment started. I wanted to check if this text was plagiarism free and for how long it will be, i.e., does ChatGPT always generate unique responses no matter how many people ask the same question?

Then I put it in Grammarly and did a quick plagiarism check. Here’s where it gets interesting in this check and the subsequent check:

1. On the day this content was generated — No hits. Grammarly said it was plagiarism free

2. One week after the content was generated — 1 direct hit. It was from a public LinkedIn post from a content marketing person, and a part of a paragraph matched verbatim. As per Grammarly, 18% was plagiarized content

3. Two weeks after the content was generated — Multiple direct hits with different public posts/articles. As per Grammarly, now the plagiarized content is 35%

Taking it one step further

So, what happens if I regenerate the content? Will that be plagiarism-free today? Let’s find out.

I asked the same question again two weeks after generating the original content. This time the response was completely different than the first time, while ChatGPT focused on all the concepts. The response was technically and coverage-wise very sound.

I did another plagiarism check, and this time it was 13% with a direct hit with two articles. I looked at both articles and was confident ChatGPT generated at least one of two, as it had the same telltale signs of ChatGPT-generated content — the structure, length, and the way the question was answered.

Repeating this process after regenerating the response a few more times, I got plagiarism percentages of 22%, 27%, and 30%, respectively.

However, asking a different question gave me a much lower hit. Some of these questions were:

• Can you please explain Cybersecurity in a non-technical way that anyone can understand? (12%)
• What I meant was that I was looking for a unique and plagiarism-free answer on what is Cybersecurity (13%)
• Follow-up question — It seems there are a lot of matches on the Internet for these answers. I was looking for a unique and plagiarism-free answer (N/A as ChatGPT gave an unexpected answer)

Answer to the follow-up question:

I apologize if my previous response did not fully meet your needs. As a language model, I do not have access to the Internet and do not reference external sources when generating responses. All of the information I provide is generated based on the knowledge and understanding I have been trained on, which includes a wide range of general knowledge and technical concepts.

If you have any specific questions or would like more information on a particular aspect of Cybersecurity, please let me know and I will do my best to assist you.

I don’t know what to say to such a humble way to put things forth.

Takeaways

1. One should refrain from using ChatGPT for creating graded assignments, research work, etc., where the originality of the content matters.

2. ChatGPT is an excellent tool for creating introductory content and can be a great learning tool in most fields.

3. The content generated may be unique and plagiarism-free at the time it was generated, but as time passes, the extent of originality (in terms of publicly available content and not in terms of what’s used for learning by the AI system itself) reduces, especially when the users are copying and pasting the responses verbatim in their posts and articles

4. It’s a reality that the answers are excellent, and OpenAI is just getting started. I’m super excited about the things to come from OpenAI and all the other players in this area.

5. The system behind it is a feat at such a level that only the most technical person can understand. A decade back, something like this could only be thought of in science fiction.

Closing Notes

I did this as a part of my research running for two weeks, without following any scientific method. This gave me a good insight into how ChatGPT and the GPT-3 work.

I find ChatGPT (and other similar systems) a great tool to learn new concepts, give live demos and explain the concepts in my classes and workshops, and I’ll be using it more often than before.

You may have a similar experience which I’d love to hear about in the comments section.