It seems lately, within the last month or so, there is a huge amount of guests on the forum.

This is a screenshot from this morning:



Has the forum suddenly become really popular, or are these bots, scraping data or whatnot?

Mostly "AI Bots" scrapping.

It is a scourge to forums.

I have two forums that I manage.


Wealth of continual fresh data and "current voice" of content, rich for "developing AI learning".

(and, yes, some site's information is more "interesting" to the efforts to mine)

A great deal of these "bots" are doing all manner of things to get around systems, rules, filters etc. and do not follow established protocols such as "robots.txt" files, sitemap.xml, meta tags, the use of encrypted verification.

All used to "instruct" bots what and where they are permitted on websites.

And "search engines" and other SEO/ranking systems (good bots) will "punish" sites that try and block bots, especially sites that do not want to be scraped as it is a tremendous load on site resources, impacts legitimate members ability to use the site without slowed interaction, admins having to deal with spurious and great deal of "membership registrations", and the constant effort on managing filtering/rules in apps to defend the site.

But these are not a bunch of people all over the world, interested in site content lurking and reading.

These are digital ticks that swarm and suck off a board, leaving it tired.

And there is big money in all of this, from the data, and the exploits, apps and methods to get around the attempts to control it.

It will get worse before it gets better.


Very appreciative of what Para does here, I can tell he busts his ass to keep this place fit to eat from.

Thanks for the good explanation sigmonkey.

Thank you monkey!