Content moderation dulu sudah susah.
Jutaan post.
Bahasa beda.
Slang.
Context.
Satire.
Hate speech.
Pornografi.
Scam.
Sekarang tambahkan AI.
Satu orang bisa membuat ribuan gambar.
Video deepfake.
Voice clone.
Artikel palsu.
Komentar otomatis.
Synthetic account.
Scale-nya berubah.
Moderasi konten AI bukan sekadar versi lebih cepat dari moderasi konten lama.
Ia membawa jenis masalah baru.
Pada Juni dan Juli 2026, Komdigi secara terbuka menyorot deepfake dan konten sintetis yang makin sulit dibedakan dari materi autentik.
Pemerintah juga mendesak platform global lebih transparan mengenai kapasitas moderasi konten mereka di Indonesia.
Januari 2026 bahkan ada pemeriksaan terhadap dugaan penyalahgunaan Grok AI di X untuk menghasilkan dan menyebarkan manipulasi foto pribadi yang bersifat asusila tanpa persetujuan.
Ini menunjukkan policy problem sudah nyata.
Bukan prediksi 2030.
Masalah pertama: volume
Dulu membuat satu video palsu berkualitas butuh skill.
Sekarang generation cost turun.
Satu aktor bisa membuat ratusan variasi.
Moderation system yang bekerja per item bisa kewalahan.
Delete satu.
Muncul sepuluh.
AI lowers production cost.
Platform perlu move upstream.
Bukan hanya takedown.
Deteksi campaign.
Account network.
Prompt abuse.
Model safeguard.
Distribution limit.
Moderation becomes system defense.
Masalah kedua: authenticity
Konten manusia bisa bohong.
Konten AI bisa benar.
Jadi label “AI” tidak sama dengan “false.”
Moderation tidak boleh otomatis menghapus karena synthetic.
Yang perlu dilihat: harm.
Impersonation.
Fraud.
Non-consensual sexual image.
False public announcement.
Manipulated evidence.
Context.
Provenance membantu.
Tapi judgment tetap.
Ini nuance.
Kalau policy = AI-generated prohibited, terlalu broad.
Creative art kena.
Parody kena.
Education kena.
Need harm-based rule.
Masalah ketiga: deepfake makin convincing
Pada Juni 2026, Wamenkomdigi Nezar Patria menekankan bahwa suara dan wajah bisa ditiru sangat halus sehingga publik sulit membedakan asli dan palsu.
Moderation challenge bukan cuma platform mendeteksi.
User juga.
Provenance.
Content credential.
Watermark.
Source verification.
Platform bisa memberi context label.
“Media ini terdeteksi dimodifikasi secara sintetis.”
Tapi detector imperfect.
False positive.
False negative.
Jangan menganggap AI detector magic.
Source-based provenance biasanya lebih kuat.
Masalah keempat: non-consensual synthetic sexual content
Ini salah satu area paling urgent.
Foto orang biasa bisa dimanipulasi.
Korban tidak pernah membuat konten.
Tidak pernah consent.
Damage real.
Reputation.
Psychological harm.
Extortion.
Policy platform harus fast.
Report channel.
Hash matching.
Re-upload prevention.
Account sanction.
Model provider juga punya responsibility mencegah generation misuse.
Komdigi pada Januari 2026 menyorot secara spesifik dugaan penggunaan Grok AI untuk manipulasi foto nyata.
Ini contoh AI safety dan content moderation bertemu.
Tidak cukup platform bilang: “user yang membuat.”
Tool itself part of ecosystem.
Masalah kelima: voice clone scam
Telepon.
“Ma, transfer sekarang.”
Suara anak.
Synthetic.
Moderation tradisional berbasis post tidak melihat.
Platform komunikasi, telco, payment, dan identity ikut.
AI content policy harus cross-sector.
Voice provenance.
Fraud signal.
Payment hold.
User education.
No single platform owns full problem.
Digital policy needs coordination.
Masalah keenam: synthetic spam
AI can create endless comment.
Fake review.
Fake profile.
Fake marketplace listing.
Moderation shifts from content quality to authenticity of behavior.
One comment harmless.
10,000 coordinated comments manipulate perception.
Platform needs network detection.
Rate limit.
Identity signal.
Bot disclosure.
Not necessarily real-name.
But automation transparency.
AI makes astroturfing cheap.
Public debate can be distorted.
Masalah ketujuh: misinformation speed
Breaking event.
Earthquake.
Election.
Health.
AI generates plausible fake before official source.
Recommendation algorithm can amplify.
Moderation has minutes.
Human fact-check takes hours.
Need friction.
Reduce virality for unverified content.
Prioritize authoritative source.
Label uncertainty.
But avoid censorship of legitimate early report.
Difficult.
Policy must accept error.
Appeal.
Transparency.
No perfect moderation.
Masalah kedelapan: language Indonesia
Global moderation model can misunderstand.
Sarcasm.
Slang.
Regional language.
“Anjir.”
“Gila.”
Context.
AI-generated disinformation can intentionally use local idiom to bypass filter.
Platform needs local training.
Human moderator.
Komdigi’s demand for transparency around moderation capacity in Indonesia becomes relevant.
How many people understand Bahasa Indonesia?
Do models support local languages?
This is infrastructure.
Not PR.
Masalah kesembilan: moderation model itself is AI
AI moderates AI.
Interesting loop.
Generator creates.
Classifier detects.
User adversarially modifies.
Classifier retrained.
Arms race.
Regulator cannot just ask: “berapa moderator manusia?”
Need understand automated system.
False positive.
Bias.
Appeal.
If AI moderation wrongly blocks legitimate speech, freedom of expression affected.
Komdigi pada Juli 2026 menegaskan moderation must balance protection and freedom of expression.
This balance becomes harder with automation.
Masalah kesepuluh: context collapse
Same image can be harmful or educational.
Deepfake shown to deceive.
Same deepfake shown in media literacy class.
Moderation model sees pixels.
Context differs.
Need signals.
Caption.
Source.
Account history.
User report.
Human review.
Fully automated moderation on nuanced content can fail.
High-impact removal maybe needs escalation.
Especially journalism.
Satire.
Political speech.
Appeal crucial.
Masalah kesebelas: AI-generated evidence
Video used as “proof.”
Police.
Court.
News.
Moderation policy intersects evidence law.
Platform cannot decide legal authenticity alone.
But can preserve provenance.
Timestamp.
Original upload.
Metadata.
Chain.
Future digital evidence needs authentication infrastructure.
AI makes “seeing is believing” obsolete.
Institutions must adapt.
Masalah kedua belas: provenance can be removed
Content credentials.
Watermark.
Great.
Then screenshot.
Crop.
Record screen.
Marker gone.
Moderation needs layered approach.
Provider marks.
Editing tool preserves.
Platform reads.
Detector estimates.
User reports.
No single defense.
Policy should not overpromise.
“Watermark solves deepfake” false.
Masalah ketiga belas: satire and parody
AI politician singing.
Meme.
Obviously fake.
Should it be labeled?
Maybe.
Should it be removed?
Probably not unless other harm.
Freedom of expression.
Label must not become punishment.
Context matters.
Platform policy should distinguish deceptive impersonation and obvious parody.
Hard in edge cases.
Human review.
Masalah keempat belas: news media will need authenticity workflow
Newsroom receives viral video.
Before publish: verify.
Source.
Metadata.
Reverse search.
Contact.
AI forensic.
This becomes standard.
Media literacy moves professional.
Publisher cannot say: “viral, jadi kami upload.”
AI content increases verification burden.
Quality journalism gets more important.
Masalah kelima belas: public officials can deny real content
This is liar’s dividend.
Because deepfake exists, someone can claim: “itu AI.”
Even when video authentic.
Moderation and provenance must protect authenticity too.
Not only detect fake.
Cryptographic capture.
Trusted source.
Chain.
If society loses ability to prove real media, problem bigger than fake content.
AI policy should support authentic provenance.
Masalah keenam belas: model provider and platform responsibility differ
Generator provider can prevent some output.
Platform can limit distribution.
User can be sanctioned.
Host can remove.
Each layer controls different risk.
Law should avoid putting all responsibility on one.
If generator creates harmful deepfake, provider has role.
If another platform amplifies to millions, platform has role.
If user intentionally targets victim, user has role.
Shared responsibility.
Masalah ketujuh belas: transparency reports need AI categories
Platform transparency report should distinguish.
Synthetic sexual abuse.
Deepfake fraud.
AI spam.
AI misinformation.
How detected?
How many appealed?
False positive?
This data helps regulator.
Without category, AI harm invisible inside “other content.”
Komdigi already asks platform for more operational transparency.
Next step could be standardized reporting.
Again, not yet a universal final obligation in Indonesia.
But policy direction logical.
Masalah kedelapan belas: moderation cost will rise
More content.
More languages.
More attacks.
Platform may rely more AI.
Smaller platform struggles.
Shared tooling.
Industry hash database.
Open safety model.
Government guidance.
Need ecosystem.
If only biggest platforms can afford compliance, competition suffers.
Risk-based requirement.
Scale matters.
High-reach platform stronger obligations.
Small forum lighter.
Masalah kesembilan belas: AI agent can moderate proactively
Interesting.
Agent can investigate account.
Check context.
Find related posts.
Escalate.
This could improve moderation.
But agent also can over-remove.
Need human oversight.
Audit.
Tool permission.
AI solution creates AI governance need.
Masalah kedua puluh: policy must evolve fast
New synthetic format appears.
3D avatar.
Real-time voice.
AR.
Rules based on “edited video” become obsolete.
Regulate deception, consent, fraud, exploitation.
Function.
Not file format.
Future-proof.
Moderasi Konten AI Akan Menjadi Masalah Kebijakan Baru because the unit of moderation is changing.
Not just a post.
Campaign.
Identity.
Provenance.
Generator.
Distribution.
AI makes content cheap and authenticity expensive.
Platform, government, media, telco, and users all need new tools.
The goal is not to make internet free of synthetic media.
Impossible and undesirable.
AI art, translation, education, and entertainment are legitimate.
The goal is to distinguish creative synthesis from harmful deception and abuse.
Ada satu problem baru lain: moderation evasion lewat generative variation. Dulu pelaku menyebarkan satu gambar yang sama. Platform bisa hash dan block re-upload. Sekarang AI bisa membuat ribuan versi yang secara visual berbeda sedikit tetapi membawa harm sama.
Hash matching menjadi kurang cukup.
Moderation membutuhkan semantic detection.
Context.
Network behavior.
Ini lebih mahal dan lebih rawan false positive.
Karena itu policy harus mendorong layered defense, bukan menuntut satu detector sempurna.
Provider model juga bisa berperan dengan abuse monitoring. Jika satu account mencoba membuat ratusan manipulasi non-consensual, signal itu penting. Tapi monitoring prompt juga menyentuh privacy. Data retention harus proporsional.
Ada trade-off lagi.
Platform ingin mendeteksi abuse.
User ingin percakapan privat.
Governance harus membatasi siapa melihat log dan untuk berapa lama.
Konten AI juga membuat takedown lintas platform makin penting. Deepfake bisa dibuat di service A, viral di B, di-repost ke C. Korban tidak seharusnya mengirim laporan dari nol ke sepuluh platform.
Trusted flagging atau interoperable abuse signal bisa membantu, khusus kategori sangat jelas seperti non-consensual intimate imagery.
Tentu safeguard diperlukan agar mekanisme lintas platform tidak menjadi alat censorship massal.
AI moderation pada akhirnya membutuhkan koordinasi lebih besar daripada moderation lama.
Generator dan distributor sekarang sering berada di perusahaan berbeda.
Harm bergerak cepat antar-platform.
Kebijakan yang hanya melihat satu service akan terus terlambat.
That is much harder than pressing delete.
And that is exactly why content moderation policy in the AI era cannot stay where it was.
Bacaan terkait
Artikel ini berada dalam Digital Policy dan mengikuti metodologi editorial Undercover.id. Format terkait tersedia di Reports dan analisis mendalam.