Bots remain active: how to protect your website in an increasingly automated world

This week, bots remain active

This week (October 28 to November 2), AI bots have continued to be the most active group on websites with BotLock enabled. Although their traffic has decreased by 8.8% compared to the previous week, it remains above the average of the last five weeks. This means that, although the attack rate has slightly moderated, bots are still trying to access content, crawl pages, and copy information to train AI models or for other automated purposes.

This is relevant to you because, even if you don't notice it, your website could be visited by thousands of bots every day. If they are not blocked, they can affect your site's performance, steal sensitive information, or even try to access protected areas like the WordPress admin panel.

What we've seen in SWPanel

This week, AI bots have represented 27.1% of total traffic on websites with BotLock enabled. Of that traffic, 12.9% has been blocked, which equals 44.8% of all blocks. This means that, although not all AI bots are malicious, a significant portion is trying to do things they shouldn't, such as copying content without authorization or attempting to access sensitive routes.

On the other hand, scraper bots, which are those that copy content to sell or reuse it, have represented 22.7% of total traffic, but only 1.9% has been blocked, which equals 5.5% of all blocks. This suggests that, although there are many bots of this type, most are not doing things that need to be blocked.

It is also important to note that MON-type bots, which are monitoring and analysis bots, have had a relatively high block rate: 36.5% of their requests have been blocked, which equals 17.5% of all blocks. This may be due to the fact that some of these bots are trying to access information they shouldn't have access to.

And in SWPanel, what?

In SWPanel, the BotLock system is active on websites that have enabled it and is automatically blocking requests that do not comply with security rules. This includes bots that try to access sensitive routes, make too many requests in a short time, or do not behave like a normal browser.

If you have BotLock enabled, your website is already protected against many of these bots. If you don't have it enabled, it's a good idea to review it, especially if your website has sensitive content or if you're concerned about security.

How we got here

The evolution of AI bot traffic over the last six weeks has been quite interesting. The average of the previous five weeks was 227, which equals 22.7% of total traffic. This week, the value was 271, which represents a +19.4% increase from the average. The maximum of the six weeks was 297 (October 21 to October 27), and the minimum was 116 (September 23 to September 29).

The general trend has been upward, suggesting that AI bots are becoming increasingly active. The linear projection for next week is 350, which would mean a +29% increase from this week. This doesn't mean it will definitely happen, but it's a trend to keep in mind.

What's happening behind the numbers

AI bots are increasing their activity because, with the growth of language models and AI tools, there is more demand for data to train those models. This means there are more bots trying to collect content from websites to use in language model training, chats, or virtual assistants.

In addition, there are bots that try to copy content to sell it or reuse it on other platforms. This can affect your website if, for example, your content is copied and published on another site, which can affect your Google ranking or your reputation.

What we can expect

If the trend continues, it is likely that AI bot traffic will continue to increase in the coming weeks. The optimistic scenario would be that the increase stabilizes and AI bots do not continue to increase their activity. The pessimistic scenario would be that the increase continues to grow and AI bots become a more serious threat to websites.

The warning signs to watch for would be:

• A sustained increase in AI bot traffic.
• An increase in the number of blocks related to AI bots.
• An increase in the number of attempts to access sensitive routes.

How it can affect your website or business

1. Loss of unique content: If your website has original content, such as articles, images, or products, and it is copied by AI bots or scrapers, it can lose its unique value and affect your Google ranking.
2. Increase in unwanted traffic: A large number of bots can affect your website's performance, which can make it load slower or even fail.
3. Security risk: If bots try to access sensitive routes, such as wp-login.php or .env, they can try to steal sensitive information or even try to access your admin panel.
4. Maintenance costs: If your website needs to be reviewed frequently to detect and block bots, it can increase the time and cost of maintenance.

How to protect yourself and prevent attacks

1. Enable BotLock: If you don't have it enabled, it's a good idea to review it. BotLock automatically blocks bots that try to do things they shouldn't do.
2. Review sensitive routes: Make sure routes like wp-login.php, .env, .git, and xmlrpc.php are protected. If you don't need them, consider disabling or blocking access.
3. Use a firewall: A firewall can help you block web attacks before they reach your application. If you don't have it enabled, it's a good idea to review it.
4. Update your CMS and plugins: If you use WordPress, make sure your CMS and plugins are up to date. Outdated plugins can have vulnerabilities that bots can exploit.
5. Use two-factor authentication (2FA): If you use WordPress, enable two-factor authentication to protect your admin panel. This makes it harder for bots to access your account.
6. Review access to your website: If you see an increase in login attempts to wp-login.php or other sensitive routes, review who is trying to access them and block suspicious IP addresses.
7. Make backups: If your website is attacked and compromised, having backups will allow you to recover it quickly.
8. Review SWPanel security alerts: If you have the SWPanel security newsletter enabled, you receive alerts when vulnerabilities that could affect your website are detected. It's a good idea to review them and act if necessary.

What SWPanel tools can help you

• BotLock: If you have it enabled, you're already blocking many bots that try to do things they shouldn't do. If you don't have it enabled, it's a good idea to review it.
• FireWall: If you have it enabled, you're already blocking web attacks before they reach your application. If you don't have it enabled, it's a good idea to review it.
• Website security analysis in Security 360: If you have it enabled, you're already detecting CMS, plugins, and versions with known issues. If you don't have it enabled, it's a good idea to review it.
• Automatic CMS updates: If you installed your website from SWPanel, you can enable automatic CMS updates to keep it updated automatically.
• Backups: If you have them enabled, you're already protecting your website in case it is attacked. If you don't have them enabled, it's a good idea to review them.
• Change PHP version: If you're using an unsupported PHP version, it's a good idea to switch to a supported version from the panel.
• Security newsletter and alerts from Security 360: If you have them enabled, you're already receiving alerts when vulnerabilities that could affect your website are detected. If you don't have them enabled, it's a good idea to review them.

Quick summary

• AI bots remain the most active group on websites with BotLock enabled.
• AI bot traffic has decreased slightly this week, but it's still above average.
• It's important to protect your website against bots that try to access sensitive routes or copy content.
• BotLock, FireWall, and automatic updates are tools that can help you protect your website.

Sources

The data for this week has been measured by us on the SWPanel platform.