AI transformation,what gets lost

AI is so nice, so easy. It can help you think and does amazing work. But it is also dangerous.  Lately I found myself staring at AI-generated text that has to be made into interactive e-learning. It was the most boring text ever. And my authoring tool, Easygenerator, has an AI as well. If I use AI on AI, what happens?

Let’s take an example. As context, I take Rwanda, because that is where I work.

I have to make an e-course on business development for the training institute Zuba, which trains rural female entepreneurs. Zuba provides me with a training manual that appears to have been generated by AI without well-designed prompts

How do I see that the manual was made by AI?

It has no spelling mistakes. The language is clear but boring. Each paragraph has the same structure: an introduction followed by six to eight bullet points and a summary, all in chunks of three-quarters of a page. There are no examples, no case studies, it is not linked to Rwanda, it is bloodless. It has multiple-choice questions at the very end.

I go back to Zuba. They say: “Yes, we paid a consultant to make this manual. It is a good manual, because it covers all subjects we need”. Hmmmm… I think we have different standards here.

What will happen if I digitise this using AI again?

If I take this AI manual and ask the AI in my authoring tool to make questions from it, the result is “model collapse”.  Model collapse happens when AI uses AI-generated input to generate more AI content.   Each AI-generated item becomes a little more general, less accurate, less diverse, less detailed, and less connected to reality. It is a bit like photocopying a photocopy and then copying it again. Details get lost, the image becomes blurry, and small smudges become larger.

The reality on the ground

Let’s assume I interview an experienced trainer from Zuba who has worked with female entrepreneurs in Rwanda for ten years. She explains: “Many women start a business selling vegetables. They are good at managing money, but they often don’t separate business money from household money. One entrepreneur, Claudine, used her business profits to pay school fees. At the end of the month, she thought her business had made no profit again, and she was ready to give up. However, she had simply mixed her household money and the money she earned from selling vegetables.”

This is a real situation, a real problem, a real person and a real decision. A good case study.

If I ask AI to summarise this interview into training material.” AI produces: “Female entrepreneurs should separate business finances from personal finances to improve financial management.” The summary is correct, but Claudine has disappeared, the school fees have disappeared, the confusion about which money has disappeared, and the emotion has disappeared. It is a lifeless text.

If I use this generated text to ask AI to make a multiple-choice question, AI writes: Why should business and personal finances be kept separate?

1. To improve financial management. ✔️
2. To increase taxes.
3. To reduce sales.
4. To hire more employees.

Technically, this is a correct question. But it is so general and concise that no learner will remember it. This is where “model collapse” appears. Nothing is wrong. But every use of AI removes some of the richness of reality.

So, I prefer to go back to the original interview and make the following question:

“Claudine runs a small vegetable business. At the end of the week, she uses the money from the cash box to pay her children’s school fees. At the end of the month, she believes her business made no profit.

What is the most likely reason?”

Now the learner has to think. The feedback text can say:

“Claudine mixed household and business expenses. The business may have been profitable. But because the money she earned from selling veggies was not recorded separately and she used the money to pay school fees, she could no longer see how the business was performing.”

The learner will probably remember Claudine much longer than the sentence “Separate business and personal finances.” A nice drawing of Claudine and her money, thinking about her daughter in a school uniform, brings the idea to life. Much better than those automated icons with arrows showing profit increase.

Can AI bring back the details?

Yes, you can use AI to bring back details. I can take an AI generated text and ask: “List the practical details that are missing. What real-life examples, common mistakes, decision points, and learner questions would make this lesson more authentic?”

Often AI enriches the text surprisingly well. AI reconstructs a realistic, pedagogically useful scenario based on general knowledge and the remaining clues.

However, AI cannot restore the lost details because they are no longer present in the source. It can create a plausible story, not recover the original one. It cannot recover what is lost.

The lesson for instructional designers

AI is a great tool, but use it wisely. AI can help to reconstruct realistic examples by creating examples based on its general knowledge. While reconstruction can produce effective learning materials, it is not the same as recovering the original real-world experiences.

For practice-based e-learning, reconstructed examples should be validated by subject-matter experts. If the source has already lost the stories, mistakes, dilemmas and context, your job as an instructional designer is to recover the missing reality by interviewing experts, asking for concrete examples from potential learners, and collecting real cases.

 

Gerry van der Hulst