"WAAHHHHH bots are wasting our bandwidth"

Because no one else is doing it--not even the media.
Post Reply
User avatar
ericbarbour
Sucks Admin
Posts: 5136
Joined: Sat Feb 25, 2017 1:56 am
Location: The ass-tral plane
Has thanked: 1371 times
Been thanked: 2115 times

"WAAHHHHH bots are wasting our bandwidth"

Post by ericbarbour » Thu Apr 03, 2025 7:33 am

https://www.theregister.com/2025/04/03/ ... bandwidth/
According to the Wikimedians, at least 65 percent of the traffic for the most expensive content served by Wikimedia Foundation datacenters is generated by bots, even though these software agents represent only about 35 percent of page views.

That's due to the Wikimedia Foundation's caching scheme which distributes popular content to regional data centers around the globe for better performance. Bots visit pages without respect to their popularity, and their requests for less popular content means that material has to be fetched from the core data center, which consumes more computing resources.
There's always a downside to being "free and open".

User avatar
Ognistysztorm
Sucks Warrior
Posts: 550
Joined: Wed Nov 09, 2022 1:39 am
Has thanked: 77 times
Been thanked: 239 times

Re: "WAAHHHHH bots are wasting our bandwidth"

Post by Ognistysztorm » Fri Apr 04, 2025 10:45 pm

ericbarbour wrote:
Thu Apr 03, 2025 7:33 am
https://www.theregister.com/2025/04/03/ ... bandwidth/
According to the Wikimedians, at least 65 percent of the traffic for the most expensive content served by Wikimedia Foundation datacenters is generated by bots, even though these software agents represent only about 35 percent of page views.

That's due to the Wikimedia Foundation's caching scheme which distributes popular content to regional data centers around the globe for better performance. Bots visit pages without respect to their popularity, and their requests for less popular content means that material has to be fetched from the core data center, which consumes more computing resources.
There's always a downside to being "free and open".
They might have bigger problems to worry about in the next coming months or years. CNN and MSNBC criticizing Wikipedia would be totally unthinkable only a few years ago.

User avatar
ericbarbour
Sucks Admin
Posts: 5136
Joined: Sat Feb 25, 2017 1:56 am
Location: The ass-tral plane
Has thanked: 1371 times
Been thanked: 2115 times

Re: "WAAHHHHH bots are wasting our bandwidth"

Post by ericbarbour » Sat Apr 05, 2025 10:23 pm

Last edited by ericbarbour on Sat Apr 05, 2025 10:24 pm, edited 1 time in total.

User avatar
ericbarbour
Sucks Admin
Posts: 5136
Joined: Sat Feb 25, 2017 1:56 am
Location: The ass-tral plane
Has thanked: 1371 times
Been thanked: 2115 times

Re: "WAAHHHHH bots are wasting our bandwidth"

Post by ericbarbour » Fri Apr 11, 2025 8:08 am

https://www.techdirt.com/2025/04/10/ai- ... b-at-risk/

This comment is especially over-the-top:
Given how the average techbro leans on the political spectrum and how Wikipedia has long been a target of the guillotinable class in general as it cannot be bought or gamed that easily, this bombardment is likely by design. If you can’t have PR people win edit wars, if you can’t write laws to remove articles on history that is inconvenient for the white nationalists in power, making the site unusuable through brute force exploitation and draining resources is the next best thing.
So, it's a CONSPIRACY now. Wikipedians normally hate conspiracy talk. But if THEY are a target (even of brainless AI scrapers), suddenly dark and horrid invisible forces are afoot to ruin their playground?

I wonder if this left-leaning ranter has connections to a WMF project.....

User avatar
ericbarbour
Sucks Admin
Posts: 5136
Joined: Sat Feb 25, 2017 1:56 am
Location: The ass-tral plane
Has thanked: 1371 times
Been thanked: 2115 times

Re: "WAAHHHHH bots are wasting our bandwidth"

Post by ericbarbour » Fri Apr 25, 2025 12:12 am

As I would expect, the WMF caved and is trying (trying!) to help AI companies by giving them a preformatted copy of WP content. Unknown how many of the raft of moneygrubbing/mercenary AI startups will actually use this, instead of ignoring nofollow tags and mindlessly scraping everything. Needless to say, the WMF is being forced to ignore the exact terms of the Creative Commons license in the process. Plus they are leaving out references.

https://gizmodo.com/wikipedia-is-making ... 2000590704
https://www.theverge.com/news/650467/wi ... e-learning

Are the Wikipoops squabbling about this yet? Well, there's a Signpost article in the making. Will there be more complaints about the WMF blowing off the CC license? Possible, but since most wikiaddicts are so firmly shoved up their own asses, not many will realize it.
https://en.wikipedia.org/wiki/Wikipedia ... _the_media

Post Reply