Feedback request: Project to show bias in wikipedia articles

[justinbrainwave]

Hi,

I want as a project to create a version of wikipedia that can show the bias it has. I'm thinking about processing it's history, then somehow displaying the bias on how the article's content was manipulated through it's history.

Some ideas I've had are:
- Have "reverts" of new content visible in some way, as long as they come from a "reputable" source. This would allow to see that there are different opinions.
- Undo any reverts coming from the "top brass" (e.g top 500 wiki editors) and performed against a "reputable" user. This would show the version of the article that the top brass disagreed with.
- Allow more reputable sources (e.g if a revision was removed due to coming from dailymail it would be now allowed)

The idea would be that if there was some kind of action to "revert" content which is factual but goes against a particular narrative, this version of wikipedia would show it. I would like suggestions and articles you think it would be useful to test against, so far I gathered these ones based on people complaining about biases:

https://en.wikipedia.org/wiki/Climate_change
https://en.wikipedia.org/wiki/COVID-19_vaccine
https://en.wikipedia.org/wiki/Barack_Obama
https://en.wikipedia.org/wiki/Mass_kill ... st_regimes
https://en.wikipedia.org/wiki/Joe_Biden
https://en.wikipedia.org/wiki/2020%E2%8 ... ial_unrest
https://en.wikipedia.org/wiki/Attempts_ ... l_election
https://en.wikipedia.org/wiki/2021_Unit ... tol_attack
https://en.wikipedia.org/wiki/Decline_o ... tern_world
https://en.wikipedia.org/wiki/Blessing_ ... n_churches

Thanks!

[Jake Is A Sellout]

Reddit is the nuts.

Look at the amazing stuff you can find there!

The point being, you get no cool points for identifying a situation you are a willing instigator of. He had a problem with me on Sucks, some sort of personality or character defect prevented him from coming to me with it the way a sensible and respectful board Admin would, and so instead he made that clown Smiley a Mod, and sat back and laughed as Smiley did what he is known for doing on forums. It had no effect on me because I am pretty much flame resistant, but once Smiley realised he couldn't drag me down to his level and was looking quite the fool, being unable to humiliate me even with the tools advantage, he pretty much set the forum on fire instead. And so now Eric is trying to come to terms with being the fire investigator who recruited an arsonist to deal with the small amount of smoke in the office bin that resulted from his own negligent practice of smoking in the office. And now he's all bitter because as it turns out, and as he surely knew all the time, Crow's can roost wherever they like and shit on whoever they want. It's just what they are. And when it comes to people needing no excuse to issue childish insults about Jimmy, and in threads which have absolutely nothing to do with Jimmy, for nothing more than a cheap thrill, Eric is the master. I kept my mouth shut about this and many of his other really embarrassing qualities since I did rather like the corner office I had at Sucks, but nor did I join in with the preferred office banter of the boss like the insecure children of Wikipediocracy do. But that's just me, a decent hard working critic who has never given anyone any reason to believe he's in this line of work for the wrong reasons or for the short haul. Eric may yet be the reason I call it quits, and if it is, I'll not be marking that sad victory for the Wikipedians/WMF with childish name calling and short term gratification, I'll be trying my hardest to honour the fine American traditions of how you settle your grievances with the PHB. I never claimed to anyone I was looking to approach Wikipedia criticism as academic or a journalist, even though I have the means and skills and knowledge to do so. But if that's what the Eric's, Jake's and Gender Lady's of this world wanted of me, if they're now regretting getting on my bad side, they should have dealt with me the way a Dean or an Editor In Chief does. Having to be nice to the lab rats, having to put up with them biting you when you're counting their diseases, having to share lab space with Section 8's, Nigerian scammers, child molesters, conspiracy theorists and special needs kids, and being talked down to and taken for granted by your boss, is not what professionals have to put up with. It is said they do the latter at the Mail, but clearly some professionals, the sharp eyed ruthless ambitious kind, will trade a little personal discomfort for being part of an award winning market leading highly influential newsroom. Eric isn't offering anyone that. Not even close. Eric was arguably the worst offender in the competition of who can run the worst outfit. He admitted himself, running shit wasn't his bag, but we see now why he kept hold of his master keys. Self interest. Even Jimmy looks good in comparison, having been brave enough to hand over what he arguably never needed to or should have. Eric's got the exit interview from Hell waiting for him, that arrogant, ignorant, useless bastard. Nobody buys a book from a man who makes the sort of basic screw-ups he does and yet seems to think he's a fucking genius. Nobody respects a man who loses an entire filing cabinet of good investigatory material in a fire of his own making, and then tries to blame the man who filled it for its loss. I don't have insurance policies or backup copies, I'm more of an old school kind of guy. You lose it through your own negligence, and you owe me a body part at the very least. This is no game for me, but it is merely a hobby not a job. Eric is the last person you would have expected not to be wise to the implications of that for his own interests given his self penned book jacket. But read a few pages, read his increasingly out of date and ill informed forum posts, and you soon realise why he can't seem to do the right thing in a crisis situation where his credentials and character are really being tested. He's a walking talking dementia case, bitter, twisted, angry, confused, cringeworthy, but most of all forgetful and unfocused, and thus quite useless. If he was ever a war hero, a Barnes Wallis, and that might just be his brain playing a cruel trick on him, he's most definitely been fighting the last war, probably the one before that, for a while now. And since he's not my pop pop, as much as I swear I think he thought he was at times, I feel not one bit of guilt or remorse for leaving him in his urine soaked bed in the care of a psychotic nurse. If me fighting said nurse pulls at his catheter, making his sunset years miserable, then fuck him. He's a Yank, so he should know the cost of not having ensured you had properly funded your health care needs. HTD.

[ericbarbour]

I want as a project to create a version of wikipedia that can show the bias it has. I'm thinking about processing it's history, then somehow displaying the bias on how the article's content was manipulated through it's history.

Just a free warning:

What you are proposing is a MASSIVE job. I tried to keep track of major editwars like climate change and US presidential bios--it was so complex and screwed up that all I could do was summarize each one and link to arbitrations etc. associated with them.

Like the man said, you are better off choosing one subject area and going thru the revisions manually, then making a summary with comparisons to talkpage, noticeboard and arbitration squabbles. That also will lead you to profile the major combatants, because it is pointless to discuss any editwar without showing the agendas of the cranks involved.

Do you know ANYTHING about climate change editwars on WP? Then I suggest you start by looking at William M. Connolley. Because he was one of the most toxic people in WP history. And obsessed with climate change, and forcing his opinion into Wikipedia content on it. And despite being desysopped and sanctioned REPEATEDLY, to this day he still has considerable control over battleground areas relating to climate change. An unbelievable asshole who has his own WIkipedia bio, despite being an utterly obscure nobody in real life. Because he has friends on Wikipedia and they keep supporting him. No matter how much he ruins content. (Repeated attempts to delete his biography failed.)

And THEN you will have to read the arbitrations dealing with his crap. There were many.
https://en.wikipedia.org/wiki/Wikipedia ... /JonGwynne
https://en.wikipedia.org/wiki/Wikipedia ... ge_dispute
https://en.wikipedia.org/wiki/Wikipedia ... _dispute_2
https://en.wikipedia.org/wiki/Wikipedia ... _Connolley
https://en.wikipedia.org/wiki/Wikipedia ... d_fusion_2
https://en.wikipedia.org/wiki/Wikipedia ... ate_change

Also don't forget: the "oversight" function available to arbitrators and certain high-ranking admins. It allows them to completely destroy all traces of individual Wikipedia edits, leaving no evidence. Oversight was used hundreds of thousands of times since it was invented in 2004. We do not know how often it was used or how it was abused to support insiders in editwars. Even THEY do not know how often it was used or where. They don't keep records. The oversight function is censorship, and they like to claim WIkipedia is NOT CENSORED. The stinking liars.

[justinbrainwave]

You can tell very powerful stories that way. If you talk about reverts and such like, your audience will only be those who already know what you're talking about (so probably already have an opinion).

I would not focus on explaining about reverts etc, the idea would be that instead of going to https://en.wikipedia.org/wiki/Barack_Obama you could go to https://en.unbiasedwikipedia.org/wiki/Barack_Obama and see the exact same wikipedia page but with different content.

What you are proposing is a MASSIVE job. I tried to keep track of major editwars like climate change and US presidential bios--it was so complex and screwed up that all I could do was summarize each one and link to arbitrations etc. associated with them.

I understand it might be technically massive, but I figure that I could start with a simple algorithim and start improving it. In your climate change example just blacklisting reverts from that user could prove to show an interesting change on the article.

Also don't forget: the "oversight" function available to arbitrators and certain high-ranking admins. It allows them to completely destroy all traces of individual Wikipedia edits, leaving no evidence. Oversight was used hundreds of thousands of times since it was invented in 2004. We do not know how often it was used or how it was abused to support insiders in editwars. Even THEY do not know how often it was used or where. They don't keep records. The oversight function is censorship, and they like to claim WIkipedia is NOT CENSORED. The stinking liars.

This is a problem, if they "deleted" anything which was really against them, then I would have nothing to work with. But I think they rely on assumption of neutrality so they wouldn't delete everything.

[ericbarbour]

This is a problem, if they "deleted" anything which was really against them, then I would have nothing to work with. But I think they rely on assumption of neutrality so they wouldn't delete everything.

You are basically correct. Far as I can tell, oversight was abused many times over the years, but only in certain "special circumstances". We never saw a case of an oversighter carefully removing thousands of categorized edits they didn't like, or ALL the edits performed by editors they hated for whatever screwed-up reason. Accounts have been banned and all their work reverted, and not all of them were simple vandals or spammers. But that's more the exception than the rule. It seems the oversight function is deliberately difficult to use, to prevent bot-driven abuse.

Also not sure it was "assumption of neutrality" as much as "it's too bloody much work". Admins in general like it when their dirty little janitorial jobs are greased for minimum effort. This is why the VAST majority of them use bots for 80-90% of the cleanup work.

As I said--any major editwar area you choose will be such a gigantic mess, reducing it to a set of neatly organized spreadsheets or graphs will be difficult. Still suggest you choose one article or a small related group of articles, and parse the edits one by one. That will be a massive job all by itself. Wikipedia uses sheer volume to hide powertripping and abuse. Take it from someone who obtained a list of all WP articles over 15k bytes in 2011, and tried to apply general subject-matter classifications to a large sample (categories are NOT reliable and not consistent). I managed to do 102,000 of them over a four-month period before giving up in disgust. It was still enough to make this chart anyway.

Chart3.png (171.18 KiB) Viewed 6097 times

Compare it to a similar chart I made of Encyclopedia Britannica articles. You will instantly see that Wikipedia is NOT a "general-subject encyclopedia" in the usual definition. It's run by nerds and fanboys and cranks. So you get a heavy bias toward nerd/fanboy content.

Chart4.png (184.71 KiB) Viewed 6094 times

[Jake Is A Sellout]

.

[justinbrainwave]

Compare it to a similar chart I made of Encyclopedia Britannica articles. You will instantly see that Wikipedia is NOT a "general-subject encyclopedia" in the usual definition. It's run by nerds and fanboys and cranks. So you get a heavy bias toward nerd/fanboy content.

That is very interesting, thanks for sharing!

I understand this is very hard, nearly impossible, which is why I like the idea. My main requirements are:

1) It should be done at scale automatically
2) It should show a version of wikipedia with less bias or marking it as obvious

From the technical side there is no worry, if you can think it I can make it happen. But from a data science perspective it needs to be simple, if I aim to start with an ML model that detects biases and removes them or marks them it will be fruitless, but anything simpler could be done as a first step.

One more question I have is, if you had to calculate a "bias score" for a user in an automated way, how would you do it?

[WWHP]

Hi,

I want as a project to create a version of wikipedia that can show the bias it has. I'm thinking about processing it's history, then somehow displaying the bias on how the article's content was manipulated through it's history.

Some ideas I've had are:
- Have "reverts" of new content visible in some way, as long as they come from a "reputable" source. This would allow to see that there are different opinions.
- Undo any reverts coming from the "top brass" (e.g top 500 wiki editors) and performed against a "reputable" user. This would show the version of the article that the top brass disagreed with.
- Allow more reputable sources (e.g if a revision was removed due to coming from dailymail it would be now allowed)

The idea would be that if there was some kind of action to "revert" content which is factual but goes against a particular narrative, this version of wikipedia would show it. I would like suggestions and articles you think it would be useful to test against, so far I gathered these ones based on people complaining about biases:

https://en.wikipedia.org/wiki/Climate_change
https://en.wikipedia.org/wiki/COVID-19_vaccine
https://en.wikipedia.org/wiki/Barack_Obama
https://en.wikipedia.org/wiki/Mass_kill ... st_regimes
https://en.wikipedia.org/wiki/Joe_Biden
https://en.wikipedia.org/wiki/2020%E2%8 ... ial_unrest
https://en.wikipedia.org/wiki/Attempts_ ... l_election
https://en.wikipedia.org/wiki/2021_Unit ... tol_attack
https://en.wikipedia.org/wiki/Decline_o ... tern_world
https://en.wikipedia.org/wiki/Blessing_ ... n_churches

Thanks!

Hi Justin,

I did an eight year case study into how majority and minority views reach a consensus in a contentious wikipedia article. Two articles that I worked on in my case study will perfectly work, however they are BLP's. Two folks not very popular on Wikipedia, Rupert Sheldrake and Deepak Chopra.

Rupert Sheldrake had majority consensus on including "biologist" in the lead section, the meta-data. His article said that for 8 years. Then a few skeptic groups came in with single purpose accounts, changed it without consensus, and since then there have been numerous attempts to correct it to no avail.

Let me k now if you think this would work and let me know if you have any more questions.

My case study is here https://rome-viharo.medium.com/aiki-wik ... dc707bdd22

[ericbarbour]

One more question I have is, if you had to calculate a "bias score" for a user in an automated way, how would you do it?

This would be a difficult question but I can share some general pointers:

a) if they focus in a very narrow subject area
b) and get into editwars and are reverted frequently
c) or, are obviously are "protected" by certain friendly administrators (the Guerrilla Skeptics and the Doc James "Medical Mafia" being two major examples, who are still active today).

Subject areas that habitually attract abuse:
--political figures, especially conservatives (watch for aggressive editwarring)
--political concepts that attract a LOT of sectarian rage, like the British Isles/Ireland or Israel-Palestine issues
--people with "outside" medical or social ideas
--anything to do with Scientology/Dianetics, Transcendental Meditation, or extremist religious cults
--anything that the notorious Chip Berlet is interested in controlling, he's basically a leftist conspiracy crank
--celebrities in the midst of a personal scandal
--Follow that epic asshole David Gerard around to see the worst of Wiki-Cranks....
--same for Guy "JzG" Chapman

There are other items but they tend to be more obscure or much lesser in severity or general interest.

[justinbrainwave]

Let me k now if you think this would work and let me know if you have any more questions.

My case study is here https://rome-viharo.medium.com/aiki-wik ... dc707bdd22

Hi! I read the article but I found the amount of links a bit overwhelming, I am interested on the aiki wiki algorithm, have you written about it or do you have have it as a proprietary product? Any algorithm to be able to find "bad faith actors" would be helpful.