• 0 Posts
  • 5 Comments
Joined 1 year ago
cake
Cake day: June 11th, 2023

help-circle

  • Wasn’t facebook also found to store images that were uploaded but not posted? This is just a resource leak . I can’t believe no one has mentioned this phrase yet. I’m more concerned about DoS attacks that fill up the instance’s storage with unused images. I think the issue of illegal content is being blown out of proportion. As long as it’s removed promptly (I believe the standard is 1 hour) when the mods/admins learn about it, there should be no liabilities. Otherwise every site that allows users to post media would be dead by now.



  • This is true. If you run the reddit-grab project directly without using the warrior (sudo docker run -d --name reddit --label=com.centurylinklabs.watchtower.enable=true --restart=unless-stopped atdr.meo.ws/archiveteam/reddit-grab --concurrent 6 yourname), you can set up to --concurrent 20, and some projects do work well with higher concurrent, but not reddit. 6 is already pushing the limit.

    I’m running reddit-grab on 25 VMs on azure (trying to burn my $200 free credit that expires in 10 days) and I can only run --concurrent 4 safely on most of them. The only VMs that can run --concurrent 6 are the ones in India, which seem to be soft-ratelimited by their higher latency anyway.


  • I think the biggest cost will be image/video storage. The text takes very little space in today’s standards. The good thing is that symmetric fibre internet connections are becoming more common so it may be possible for members of the instance to contribute unused disk space to help with its image/video storage. This plus limiting the image/video sizes (and maybe forbidding video uploads altogether) will allow the instances to scale with user count.