A Telegram user who advertises their services on Twitter will create an AI-generated pornographic image of anyone in the world for as little as $10 if users send them pictures of that person. Like many other Telegram communities and users producing nonconsensual AI-generated sexual images, this user creates fake nude images of celebrities, including images of minors in swimsuits, but is particularly notable because it plainly and openly shows one of the most severe harms of generative AI tools: easily creating nonconsensual pornography of ordinary people.

  • echo64@lemmy.world
    link
    fedilink
    English
    arrow-up
    89
    arrow-down
    31
    ·
    8 months ago

    I hate this: “Just accept it women of the world, accept the abuse because it’s the new normal” techbro logic so much. It’s absolutely hateful towards women.

    We have legal and justice systems to deal with this. It is not the new normal for me to be able to make porn of your sister, or mother, or daughter. Absolutely fucking abhorrent.

    • AquaTofana@lemmy.world
      link
      fedilink
      English
      arrow-up
      55
      arrow-down
      6
      ·
      8 months ago

      I don’t know why you’re being down voted. Sure, it’s unfortunately been happening for a while, but we’re just supposed to keep quiet about it and let it go?

      I’m sorry, putting my face on a naked body that’s not mine is one thing, but I really do fear for the people whose likeness gets used in some degrading/depraved porn and it’s actually believable because it’s AI generated. That is SO much worse/psychologically damaging if they find out about it.

    • brbposting@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      17
      arrow-down
      2
      ·
      8 months ago

      It’s unacceptable.

      We have legal and justice systems to deal with this.

      For reference, here’s how we’re doing with child porn. Platforms with problems include (copying from my comment two months ago):

      Ill adults and poor kids generate and sell CSAM. Common to advertise on IG, sell on TG. Huge problem as that Stanford report shows.

      Telegram got right on it (not). Fuckers.

    • cley_faye@lemmy.world
      link
      fedilink
      English
      arrow-up
      7
      arrow-down
      3
      ·
      8 months ago

      How do you propose to deal with someone doing this on their computer, not posting them online, for their “enjoyment”? Mass global surveillance of all existing devices?

      It’s not a matter of willingly accepting it; it’s a matter of looking at what can be done and what can not. Publishing fake porn, defaming people, and other similar actions are already (I hope… I am not a lawyer) illegal. Asking for the technology that exists, is available, will continue to grow, and can be used in a private setting with no witness to somehow “stop” because of a law is at best wishful thinking.

      • Ookami38@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        4
        ·
        8 months ago

        There’s nothing to be done, nor should be done, for anything someone individually creates, for their own individual use, never to see the light of day. Anything else is about one step removed from thought policing - afterall what’s the difference between a personally created, private image and the thoughts on your brain?

        The other side of that is, we have to have protection for people who this has or will be used against. Strict laws regarding posting or sharing material. Easy and fast removal of abusive material. Actual enforcement. I know we have these things in place already, but they need to be stronger and more robust. The one absolute truth with generative AI, versus Photoshop etc is that it’s significantly faster and easier, thus there will likely be an uptick in this kind of material, thus the need for re-examining current laws.

      • Jrockwar@feddit.uk
        link
        fedilink
        English
        arrow-up
        13
        arrow-down
        5
        ·
        edit-2
        8 months ago

        And so is straight male-focused porn. We men seemingly are not attractive, other than for perfume ads. It’s unbelievable gender roles are still so strongly coded in 20204. Women must be pretty, men must buy products where women look pretty in ads. Men don’t look pretty and women don’t buy products - they clean the house and care for the kids.

        I’m aware of how much I’m extrapolating, but a lot of this is the subtext under “they’ll make porn of your sisters and daughters” but leaving out of the thought train your good looking brother/son, when that’d be just as hurtful for them and yourself.

        • lud@lemm.ee
          link
          fedilink
          English
          arrow-up
          5
          ·
          8 months ago

          Or your bad looking brother or the bad looking myself.

          Imo people making ai fakes for themselves isn’t the end of the world but the real problem is in distribution and blackmail.

          You can get blackmailed no matter your gender and it will happen to both genders.

      • echo64@lemmy.world
        link
        fedilink
        English
        arrow-up
        9
        arrow-down
        16
        ·
        8 months ago

        Sorry if I didn’t position this about men. They are the most important thing to discuss and will be the most impacted here, obviously. We must center men on this subject too.

        • Thorny_Insight@lemm.ee
          link
          fedilink
          English
          arrow-up
          15
          arrow-down
          6
          ·
          8 months ago

          Pointing out your sexism isn’t saying we should be talking about just men. It you whose here acting all holy while ignoring half of the population.

          • echo64@lemmy.world
            link
            fedilink
            English
            arrow-up
            5
            arrow-down
            17
            ·
            8 months ago

            Yes yes, #alllivesmatter amiirte? We just ignore that 99.999% of the victims will be women, just so we can grandstand about men.

    • SharkAttak@kbin.social
      link
      fedilink
      arrow-up
      9
      arrow-down
      9
      ·
      8 months ago

      It’s not normal but neither is new: you already could cut and glue your cousin’s photo on a Playboy girl, or Photoshop the hot neighbour on Stallone’s muscle body. Today is just easier.

      • echo64@lemmy.world
        link
        fedilink
        English
        arrow-up
        9
        arrow-down
        6
        ·
        8 months ago

        I don’t care if it’s not new, no one cares about how new it is.