A suggestion for echomail hub function.

Had a node that had a BOT malfunction that was sending out around multiple messages per minute to the FSX_BOT echo. All messages had different MSGID and were essentially different in body content.

It would be good if Mystic could detect the flooding of an echo by a node and if a threshold is reached stop the messages from continuing to be processed. Perhaps an auto delink from echo if x messages posted within x minutes plus a netmail alert sent to node + hub admin?

Dunno... but thought I'd pass this one along, not sure how other tossers handle such things. We ended up with 8570 messages posted before it was caught and manually delinked.

Best, Paul

--- Mystic BBS v1.12 A47 2021/09/29 (Linux/64)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (3:770/100)

On 08 Nov 2021, Paul Hayton said the following...

Dunno... but thought I'd pass this one along, not sure how other tossers handle such things. We ended up with 8570 messages posted before it was caught and manually delinked.

Another "nice to have" would be the ability to select multiple messages to delete at once, similar to how you can use tab to select messages bases to globally update or delete.


Jay

... The brain is as strong as its weakest think.

--- Mystic BBS v1.12 A47 2021/11/06 (Raspberry Pi/32)
* Origin: Northern Realms (1:229/664)

It would be good if Mystic could detect the flooding of an echo by a
node and if a threshold is reached stop the messages from continuing to
be processed. Perhaps an auto delink from echo if x messages posted
within x minutes plus a netmail alert sent to node + hub admin?

I am open to trying to build something like this, but I am not really sure how well it would work. If a node spams connections to you that can be covered by the BINKP auto blocking, but I realize that doesn't really address this specific issue.

When talking about importing a packet it would be tough to determine when its a flood if the message content is different each time. If we put a threshold in for messages then it could backfire in situations where people are trying to do rescans or maybe even for systems who only poll daily or whenever they feel like reading (so they tend to get mail in larger clumps)...

I suppose in the case of a hub though, there wouldn't be many instances of a rescan coming into the hub, so maybe the threshold could work but just be disabled by default? Or set to something really high.

The tracking would be a bit difficult or at the very least annoying to do and it would probably slow down tossing speed a bit since it would have to compare every single message being tossed and it would have to save this sort of data for every node between MUTIL execution.

How did these messages come in? Where they connecting to you every minute to send a new message or did it come in clumps every so often? I think having a clearer picture might help me think it through a little better.

... Some people have no idea what they're doing, and are really good at it!

--- Mystic BBS v1.12 A47 2021/11/06 (Windows/32)
* Origin: Sector 7 * Mystic WHQ (1:129/215)

On 08 Nov 2021 at 08:10a, g00r00 pondered and said...

How did these messages come in? Where they connecting to you every
minute to send a new message or did it come in clumps every so often? I think having a clearer picture might help me think it through a little better.

thanks for the considered reply, apologies for my delay in reply, busy days on the work front also. I will look and see what I can find then email you with what I find.

--- Mystic BBS v1.12 A47 2021/09/29 (Linux/64)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (3:770/100)

On 08 Nov 2021 at 08:10a, g00r00 pondered and said...

How did these messages come in? Where they connecting to you every
minute to send a new message or did it come in clumps every so often? I think having a clearer picture might help me think it through a little better.

Have just dug back into the HUB logs and emailed you some files. Essentially it looks like one packet being processed by MUTIL and it contained approx 5 messages for the echomail area within it. The next packet arrived and was processed around 2 mins later and so on.

I am open to trying to build something like this, but I am not really
sure how well it would work. If a node spams connections to you that

Thanks for looking / considering it. If it proves overly problematic and does not lead to something you want to add I'm understanding of that too.

can be covered by the BINKP auto blocking, but I realize that doesn't really address this specific issue.

No, in this case it's the tossing frequency rather than the mailer frequency.

When talking about importing a packet it would be tough to determine
when its a flood if the message content is different each time. If we
put a threshold in for messages then it could backfire in situations
where people are trying to do rescans or maybe even for systems who only poll daily or whenever they feel like reading (so they tend to get mail
in larger clumps)...

Agreed, as in the examples above, it's nodes wanting to toss a large amount of packets and messages within a short space of time, such as a rescan etc.

I suppose in the case of a hub though, there wouldn't be many instances
of a rescan coming into the hub, so maybe the threshold could work but just be disabled by default? Or set to something really high.

Yes I agree. From the HUB POV it's about the node sending something in such that the quantity of messages posted over a defined short period of time is leading to issues for others connected to the HUB and it's echos.

The tracking would be a bit difficult or at the very least annoying to
do and it would probably slow down tossing speed a bit since it would
have to compare every single message being tossed and it would have to save this sort of data for every node between MUTIL execution.

Instead of inspecting message content could a frequency counter be implemented such that if x messages from a node were being tossed over x period then an issue could be flagged and an action(s) taken?

As I sit and ponder this it's really about the number of incoming messages to be tossed to a given echo or echos from a node over a short time period.

If you did end up adding something I'm not sure if such a HUB setting should be set at a high level to apply to all nodes and/or if it could/should be set at a echomail area level? Gotta ponder that.

--- Mystic BBS v1.12 A47 2021/11/06 (Linux/64)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (3:770/100)

On 13 Nov 2021 at 10:07a, Paul Hayton pondered and said...

On 08 Nov 2021 at 08:10a, g00r00 pondered and said...

How did these messages come in? Where they connecting to you every minute to send a new message or did it come in clumps every so often? think having a clearer picture might help me think it through a littl better.

Have just dug back into the HUB logs and emailed you some files. Essentially it looks like one packet being processed by MUTIL and it contained approx 5 messages for the echomail area within it. The next packet arrived and was processed around 2 mins later and so on.

Hi there

Just circling back to this one, wondered if what I sent you was of any help and if you had any further thoughts on this one?

--- Mystic BBS v1.12 A47 2021/11/06 (Linux/64)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (3:770/100)

Just circling back to this one, wondered if what I sent you was of any help and if you had any further thoughts on this one?

Hey there, not yet.

Its probably going to be a bit time consuming to do so I have to get some time aside to experiment with how it might work. I've barely been touching any code lately unfortunately (just little touch ups mostly).

Almost all of my time is spent just responding to stuff. :(

It might not be until the new year before things ease up for me. I hope I can get more time before that to see what could be done. First goal will be to get A47 out the door before I add anything new to the tosser. For now I am trying to handle support issues/reported issues as much as I can so I can accomplish that goal.

... Kilometers are shorter than miles. Save gas, take your trip in kilometers

--- Mystic BBS v1.12 A47 2021/11/18 (Windows/64)
* Origin: Sector 7 * Mystic WHQ (1:129/215)

Hello g00r00,

On Fri Nov 19 2021 09:55:40, you wrote to Paul Hayton:

Hey there, not yet.

Its probably going to be a bit time consuming to do so I have to get
some time aside to experiment with how it might work. I've barely been touching any code lately unfortunately (just little touch ups mostly).

To add to this, if something is indeed done with this (I can't imagine this won't cause future problems and/or missing echomail), have the threshold set so high by default that it will not affect the way things currently work. Also maybe have some big bold red text pop up warning people that don't know wtf they're doing when you try to edit/change anything regarding it.

If I remember right, I don't think the problematic system in this case was a Mystic system? The system that was sending all of this regurgitated and/or stuck or looped echomail should be where the issue is fixed, in my opinion. Especially since it seems to have only happened once, and from one specific system.

Regards,
Nick

... "Take my advice, I don't use it anyway."
--- GoldED+/LNX 1.1.5-b20210705
* Origin: thePharcyde_ distribution system (Wisconsin) (1:154/10)

If I remember right, I don't think the problematic system in this case
was a Mystic system? The system that was sending all of this
regurgitated and/or stuck or looped echomail should be where the issue
is fixed, in my opinion. Especially since it seems to have only happened once, and from one specific system.

Yeah its a rare situation thats for sure, and this isn't really something that any other tossers support doing as far as I know. I do want to add a lot more statistical tracking without requiring log parsing though, so I am keeping an open mind for this...

If I can fit it into the tosser without going way out of my way, and the performance impact isn't terrible, then I don't see why I couldn't add something in.

Its just finding the time!

... Youth is glorious, but it isn't a career

--- Mystic BBS v1.12 A47 2021/11/18 (Windows/64)
* Origin: Sector 7 * Mystic WHQ (1:129/215)

On 19 Nov 2021 at 09:55a, g00r00 pondered and said...

Its probably going to be a bit time consuming to do so I have to get
some time aside to experiment with how it might work. I've barely been touching any code lately unfortunately (just little touch ups mostly). Almost all of my time is spent just responding to stuff. :(
It might not be until the new year before things ease up for me. I hope

Hey there :)

It's all good, no stress regarding looking at this one, if you find time in 2022 then that would be great. I think it would be something worth exploring so that at some high level there was logic that could negate such issues if Mystic was tossing such stuff from other nodes.

I accept the comments from others about how a node should be the main area of focus to fix any issue if it's the one generating the problem. My perspective was simply that if a HUB aka tosser could have some logic built in to it such that it could assist in negating echomail area flooding onwards to others, then that would be a useful thing to implement as well.

I can get more time before that to see what could be done. First goal will be to get A47 out the door before I add anything new to the tosser. For now I am trying to handle support issues/reported issues as much as
I can so I can accomplish that goal.

I'm looking forward to seeing A47 in the public domain, I'll have some videos to work on / update when it does :)

Best, Paul

--- Mystic BBS v1.12 A47 2021/11/06 (Linux/64)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (3:770/100)