Hello to you!
I think timestamps of files would be one of the easier things, and try to track back to postings and comments that references the upload… ideally the logged-in account (which is the standard install of lemmy, only logged-in users can upload to pictrs)
Yes. odd how people think sharing CSAM is why people would post here, instead of actually tracking down and prosecuting those sharing CSAM. Details about the users who sharedl CSAM content, such as timestamps - would help identify the offenders for prosecution.
It sounds like you’re encouraging people to share CSAM images found, which is obviously not the intent of this tool.
Yes, that is in fact the context.
Context: "which is obviously not the intent of this tool. "
it is not my intent to share the images, nor is it the context of the tool… Sharing details about the users, timestamps - would be the obvious context.
I hope people share the positive hits of CSAM and see how widespread the problem is…
DRAMTIC EDIT: the records lemmy_safety_local_storage.py identifies, not the images! @[email protected] seems to think it “sounds like” I am ACTIVELY encouraging the spreading of child pornography images… NO! I mean audit files, such as timestamps, the account that uploaded, etc. Once you have the timestamp, the nginx logs from a lemmy server should help identify the IP address.
and avoiding link rot
Lemmy seems built to destroy information, rot links. Unlike Reddit has been for 15 years, when a person deletes their account Lemmy removes all posts and comments, creating a black hole.
Not only are the comments disappeared from the person who deleted their account, all the comments made by other users disappear on those posts and comments.
Right now, a single user just deleting one comment results in the entire branch of comment replies to just disappear.
Installing an instance was done pretty quickly… over 1000 new instances went online in June because of the Reddit API change. But once that instance goes offline, all the communities hosted there are orphaned and no cleanup code really exists to salvage any of it - because the whole system was built around deleting comments and posts - and deleting an instance is pretty much a purging of everything they ever created in the minds of the designers.
yha, what do people think the FBI is for… this isn’t crazy. They can get access to ISP logs, VPN provider logs, etc.
CSAM (Child Sexual Assault Material) posts
The federal governments of several nations should be in pursuit of this, and IP addresses and specific time logs shared.
Nothing like a little bit of corporate sabotage!
The software developers who created Lemmy openly criticize systems of government and economics. These are nation-state battlegrounds too. The barrier to entrance is very low, as Lemmy doesn’t even do routine tracking of account creation, rate-limiting alone isn’t really defensive. 15 years ago sites like Reddit had major vote manipulation detection logic behind the scenes. This is pretty much unleashed playground for a lot of known tactics.
@[email protected] and @[email protected]
I added some self-awareness of concurrency to the Lemmy API Rust logic: https://github.com/LemmyNet/lemmy/pull/3805/files - this should allow a more standard in-band response to concurrency slow-response server overloads. Keywords (‘woodstocksnoopy’ as watchdog, and even more alarming keyworld to find in your logs: ‘SITE_OVERLOAD’) were added to the logging to make it explicit when this is happening.
This post you made, the language you set on it is “中文”.
comments are hidden on some posts I’ve seen, like it says there are 8 but I see 2.
Lemmy has bug in counting too. Comments are often missing because of replication issues between servers. But the most common issue I’v seen with inflated comment numbers is edits being counted as new comments.
The bugs in Lemmy are such that you don’t even need to touch a server for it to be vulnerable. Cloudflare does not defend against such mistakes. Other servers can trigger deep PostgreSQL logic problems within Lemmy. Growing pains, a lot of the federation code was never tested, and today’s crash is due to a logic issue with lemmy_server mistakenly updating 1700 servers it knows of through federation for a delete instead of the 1 local server.
Most of these ‘attacks’ are targeted at the database
A major PostgreSQL performance issue, logic mistake, was discovered today in lemmy_server and is an easy fix. Details: https://lemmy.world/post/2008987
Good to see a heavy production server taking on the scaling issues. Thank you! To discuss Lemmy performance issues, there is a dedicated community: [email protected]
Edit still see some performance issues. Needs more troubleshooting
Federation overheard is putting a lot of load on servers. Creating one task for every single post, comment, and vote in RAM-only queue… pending changes: https://github.com/LemmyNet/lemmy/pull/3466
Federation also does a lot of live HTTP connects to other peers. It looks up users for messages. The whole design is very resource intensive, one single vote, comment, post at a time. There is also a lot of boilerplate JSON overhead in sending something as simple as a single vote.
we created extra lemmy containers to spread the load. (And extra lemmy-ui containers).
Is Rust HTTP server running into thread limits? database connection pooling? All kinds of internal questions bout that solution.
My lemmy server is also running the same versions are you are now. I was getting timeouts with lemmy.world federation so I recompiled lemmy_server with timeout changes:
diff --git a/crates/utils/src/lib.rs b/crates/utils/src/lib.rs
index e5d07db2c..e2c592d82 100644
--- a/crates/utils/src/lib.rs
+++ b/crates/utils/src/lib.rs
@@ -18,7 +18,7 @@ use std::time::Duration;
pub type ConnectionId = usize;
-pub const REQWEST_TIMEOUT: Duration = Duration::from_secs(10);
+pub const REQWEST_TIMEOUT: Duration = Duration::from_secs(13);
#[macro_export]
macro_rules! location_info {
diff --git a/src/lib.rs b/src/lib.rs
index cc77ca48f..45c621a7c 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -37,7 +37,7 @@ use tracing_subscriber::{filter::Targets, layer::SubscriberExt, Layer, Registry}
use url::Url;
/// Max timeout for http requests
-pub(crate) const REQWEST_TIMEOUT: Duration = Duration::from_secs(10);
+pub(crate) const REQWEST_TIMEOUT: Duration = Duration::from_secs(16);
/// Placing the main function in lib.rs allows other crates to import it and embed Lemmy
pub async fn start_lemmy_server() -> Result<(), LemmyError> {
I think a static page page explaining to users that the server is being upgraded would be best.
Beehaw.org did this, their nginx seems configured to return a nice error message.
It is not even a mistake, it’s some pretty mind-fucked up on part of @[email protected] to jump to such a conclusion. crap