There's always a downside to being "free and open".According to the Wikimedians, at least 65 percent of the traffic for the most expensive content served by Wikimedia Foundation datacenters is generated by bots, even though these software agents represent only about 35 percent of page views.
That's due to the Wikimedia Foundation's caching scheme which distributes popular content to regional data centers around the globe for better performance. Bots visit pages without respect to their popularity, and their requests for less popular content means that material has to be fetched from the core data center, which consumes more computing resources.
"WAAHHHHH bots are wasting our bandwidth"
-
- Sucks Admin
- Posts: 5136
- Joined: Sat Feb 25, 2017 1:56 am
- Location: The ass-tral plane
- Has thanked: 1371 times
- Been thanked: 2115 times
"WAAHHHHH bots are wasting our bandwidth"
https://www.theregister.com/2025/04/03/ ... bandwidth/
-
- Sucks Warrior
- Posts: 550
- Joined: Wed Nov 09, 2022 1:39 am
- Has thanked: 77 times
- Been thanked: 239 times
Re: "WAAHHHHH bots are wasting our bandwidth"
They might have bigger problems to worry about in the next coming months or years. CNN and MSNBC criticizing Wikipedia would be totally unthinkable only a few years ago.ericbarbour wrote: ↑Thu Apr 03, 2025 7:33 amhttps://www.theregister.com/2025/04/03/ ... bandwidth/
There's always a downside to being "free and open".According to the Wikimedians, at least 65 percent of the traffic for the most expensive content served by Wikimedia Foundation datacenters is generated by bots, even though these software agents represent only about 35 percent of page views.
That's due to the Wikimedia Foundation's caching scheme which distributes popular content to regional data centers around the globe for better performance. Bots visit pages without respect to their popularity, and their requests for less popular content means that material has to be fetched from the core data center, which consumes more computing resources.
-
- Sucks Admin
- Posts: 5136
- Joined: Sat Feb 25, 2017 1:56 am
- Location: The ass-tral plane
- Has thanked: 1371 times
- Been thanked: 2115 times
Re: "WAAHHHHH bots are wasting our bandwidth"
The WAAAHHHHH continues.....
https://arstechnica.com/information-tec ... surges-50/
https://techcrunch.com/2025/04/02/ai-cr ... -surge-50/
https://www.newscientist.com/article/24 ... wikipedia/
And the Delhi High Court continues to poke the (fake) hornet's nest
https://www.reuters.com/world/india/wik ... 025-04-04/
¯\_(ツ)_/¯
https://arstechnica.com/information-tec ... surges-50/
https://techcrunch.com/2025/04/02/ai-cr ... -surge-50/
https://www.newscientist.com/article/24 ... wikipedia/
And the Delhi High Court continues to poke the (fake) hornet's nest
https://www.reuters.com/world/india/wik ... 025-04-04/
¯\_(ツ)_/¯
Last edited by ericbarbour on Sat Apr 05, 2025 10:24 pm, edited 1 time in total.
-
- Sucks Admin
- Posts: 5136
- Joined: Sat Feb 25, 2017 1:56 am
- Location: The ass-tral plane
- Has thanked: 1371 times
- Been thanked: 2115 times
Re: "WAAHHHHH bots are wasting our bandwidth"
https://www.techdirt.com/2025/04/10/ai- ... b-at-risk/
This comment is especially over-the-top:
I wonder if this left-leaning ranter has connections to a WMF project.....
This comment is especially over-the-top:
So, it's a CONSPIRACY now. Wikipedians normally hate conspiracy talk. But if THEY are a target (even of brainless AI scrapers), suddenly dark and horrid invisible forces are afoot to ruin their playground?Given how the average techbro leans on the political spectrum and how Wikipedia has long been a target of the guillotinable class in general as it cannot be bought or gamed that easily, this bombardment is likely by design. If you can’t have PR people win edit wars, if you can’t write laws to remove articles on history that is inconvenient for the white nationalists in power, making the site unusuable through brute force exploitation and draining resources is the next best thing.
I wonder if this left-leaning ranter has connections to a WMF project.....
-
- Sucks Admin
- Posts: 5136
- Joined: Sat Feb 25, 2017 1:56 am
- Location: The ass-tral plane
- Has thanked: 1371 times
- Been thanked: 2115 times
Re: "WAAHHHHH bots are wasting our bandwidth"
As I would expect, the WMF caved and is trying (trying!) to help AI companies by giving them a preformatted copy of WP content. Unknown how many of the raft of moneygrubbing/mercenary AI startups will actually use this, instead of ignoring nofollow tags and mindlessly scraping everything. Needless to say, the WMF is being forced to ignore the exact terms of the Creative Commons license in the process. Plus they are leaving out references.
https://gizmodo.com/wikipedia-is-making ... 2000590704
https://www.theverge.com/news/650467/wi ... e-learning
Are the Wikipoops squabbling about this yet? Well, there's a Signpost article in the making. Will there be more complaints about the WMF blowing off the CC license? Possible, but since most wikiaddicts are so firmly shoved up their own asses, not many will realize it.
https://en.wikipedia.org/wiki/Wikipedia ... _the_media
https://gizmodo.com/wikipedia-is-making ... 2000590704
https://www.theverge.com/news/650467/wi ... e-learning
Are the Wikipoops squabbling about this yet? Well, there's a Signpost article in the making. Will there be more complaints about the WMF blowing off the CC license? Possible, but since most wikiaddicts are so firmly shoved up their own asses, not many will realize it.
https://en.wikipedia.org/wiki/Wikipedia ... _the_media