CYBERJAYA: The misuse of artificial intelligence (AI) to create sexually explicit images, particularly of children, is on the rise, prompting calls for stronger safeguards and closer parental supervision.
Multimedia Content Forum of Malaysia (CMCF) chief executive officer Mediha Mahmood said AI tools could be used to turn images of people into sexually explicit materials or deepfake videos in inappropriate situations.
"It is not an authentic image. It is a manipulated image or deepfake video altered by AI to depict the victim in a sexual or inappropriate manner," she said recently.
Mediha said the growing use of AI tools by children is another concern, as unrestricted applications and websites may make it easier to create illicit content.
"The most glaring issue is the proliferation of unrestricted apps, websites and open-source tools that can be repurposed for these malicious purposes.
"There are so many of them," she said.
She said easy accessibility of these applications has made it difficult to monitor or control the use of AI platforms.
"These AI-generation platforms cannot always be fully monitored or controlled. Most concerningly, they are accessible to anyone with a connected device, without verifying whether the user is an adult or a child.
"Access remains largely unrestricted for minors and other vulnerable groups," she said.
Mediha said distributing sexually explicit material involving children is an offence under the Sexual Offences Against Children Act 2017, even if the person circulating it is a minor.
[[nid:1557873]]
"This is a critical point that must be impressed upon parents, guardians, teachers and anyone who interacts regularly with children."
She said Malaysians have embraced AI for creativity and efficiency, but malicious users have taken advantage of the technology.
"Like any other service or application, bad actors will exploit and manipulate it.
"We see amazing uses of AI for creativity and efficiency. But because it is available to everyone, we also see bad actors using it for nefarious purposes," she said.
Mediha said existing laws, including the Penal Code, the Communications and Multimedia Act 1998 and the Online Safety Act 2026, may be used to address AI misuse.
However, gathering digital evidence is a challenge for enforcement agencies.
She called on those who have fallen victim to AI manipulation or those aware of such incidents to lodge reports with the police and relevant agencies.
She said even after the online content has been taken down or deleted, this does not mean that the evidence can no longer be recovered.
"People often assume that once a link is deleted, nothing more can be done.
"But in discussions with enforcement agencies, such as the police and the Malaysian Communications and Multimedia Commission (MCMC), there are technical avenues to recover links and trace digital forensic evidence.
"People should not assume an incident cannot be reported. Reports can and should be filed, and action will be taken," she said.
Mediha said digital platforms should adopt the "Safety by Design" and "Safety by Default" principles.
They include restricting certain image-manipulation features for minors and requiring verifiable consent before images can be altered.
While social media platforms have been working with MCMC to comply with regulatory frameworks and licensing requirements, she said collaboration with generative AI providers remains limited.
"From CMCF's perspective, there has not yet been meaningful collaboration, largely because engaging these independent AI developers and providers is quite difficult."
Mediha said the problem is not a lack of legislation.
"Rather, it is a lack of awareness among parents who do not fully grasp the potential harm their children face.
"It is fundamentally an advocacy and digital literacy issue," she said.
She cited MCMC's Internet Selamat (Safe Internet) campaign as an initiative to educate parents about online risks, platform safety features and how to configure them.
"Parents need to understand the threats, learn about the safety features offered by platforms and know how to configure them," she said.
Mediha wants parents to supervise their children's use of AI applications, particularly when providing them with Internet-connected devices or devices with recording capabilities.
"Whether they like it or not, if parents provide Internet-connected devices or devices with recording capabilities to their children, they must ensure the usage is supervised," she said.
She added that the government's stand of not allowing children under 16 to open social media accounts should serve as a benchmark for other online services.
Mediha said preventing AI misuse requires shared responsibility among regulators, enforcement agencies, digital platforms, parents and the wider community.
"We also need everyone to help these children understand the grave dangers of using AI to generate inappropriate imagery of themselves or their peers and sharing it."
© New Straits Times Press (M) Bhd