On each page, they describe, in detail, exactly how they come to their conclusions.
Comment on Why are people downvoting the MediaBiasFactChecker not?
FuglyDuck@lemmy.world 1 month agoBecause their methodology is nothing but buzzwords:
The primary aim of our methodology is to systematically evaluate the ideological leanings and factual accuracy of media and information outlets. This is achieved through a multi-faceted approach that incorporates both quantitative metrics and qualitative assessments in accordance with our rigorously defined criteria.
Despite apparently having “rigorously defined criteria”, they don’t actually say what they are.
finley@lemm.ee 1 month ago
FuglyDuck@lemmy.world 1 month ago
here’s their definition of what’s a left or right bias
It’s pretty fucking arbitrary.
Additionally, their methodology is a bunch of gibberish and buzz words. that they explain their justification on each article is inadequate. For example, Al jazeera is dinged for using “negative emotion” words like “Deadly”.
Deadly might invoke a certain kind of emotion. but it’s also the simplest way to describe an attack in which some one dies. Literally every news service will use “deadly attack” if people are dying, regardless if it’s an attack by terrorists, or by cackling baboons. (or indeed not even an attack. for example ‘Deadly wildfire’ or ‘deadly hurricane’.) the application of using that as an example is extremely arbitrary, on a case by case basis.
finley@lemm.ee 1 month ago
Now you’re just repeating yourself.
FuglyDuck@lemmy.world 1 month ago
Okay.
Take their methodology.
Work through it.
You can’t because most of the “rigorous definitions “ aren’t shared.
You still haven’t explained what “factually consistent” means in a method that’s repeated and able to be applied regularly.
Their methodology as posted is far too vague to adequately consider their ability to provide consistent neutral ratings.
How are “loaded” words evaluated? Is there a table of words that are considered “loaded”? Personal feeling? We don’t know. We know what some of them are, since they’re mentioned on specific articles.
But that isn’t a consistent or “rigorously defined” criteria. So what is the “rigorously defined criteria”- and why is that not published?
Do you not see how that’s ripe for abuse?
just2look@lemm.ee 1 month ago
They literally publish their methodology and scoring system.
mediabiasfactcheck.com/methodology/
So they do say exactly what their criteria is, and how it is scored. And none of that is buzz words, it’s just a summary that fit in a few sentences. And you can look at the full methodology if you want more than just that small bullet description.
I’m not saying that you have to agree with their scoring, or that it is necessarily accurate. I just think if you’re going to critique a thing, you should at least know what you’re critiquing.
protist@mander.xyz 1 month ago
But what even is this false left-right, liberal-conservative, Democrat-Republican one-dimensional scale? The first thing they state on this page is that all this is inherently subjective. Who is MBFC to determine where the middle of this scale exists? If people want to seek out their opinion, that’s fine, but this is inherently a subjective opinion about what constitutes “left center” vs “center,” for example. I don’t get how MBFC deserves their opinion on every news post.
Also the formatting of the bot is awful as displayed on most Lemmy apps. On mine it’s a giant wall of text. Other posts/bots don’t look bad, just this one.
just2look@lemm.ee 1 month ago
They cover what they consider left and right. This way you can judge whether it aligns with what you believe. And it allows you to interpret their results even if they don’t follow the same spectrum you do.
And if you know of a way to discuss political spectrum without subjectivity I would love to hear it. Even if you don’t use a 2d spectrum, it’s still subjective. Just subjective with additional criteria.
mediabiasfactcheck.com/left-vs-right-bias-how-we-…
protist@mander.xyz 1 month ago
Of course that doesn’t exist, my point is why does this specific subjective opinion get promoted on here?
Artisian@lemmy.world 1 month ago
Bravo for bringing the notes. On a first glance, some of these feel like they require subjectivity (like, do we really believe the political spectrum is 1d?), but I agree I could run the computation myself from this.
just2look@lemm.ee 1 month ago
There is definitely some subjectivity. Language isn’t something that is easily parsed and scored. That is why they give examples on the actual report about the kind of biased language they saw, or whatever other issues led to the score given.
I don’t think they mean for their website to be the end all bias resource. More of a stepping off point for you to make your own judgments.
FuglyDuck@lemmy.world 1 month ago
Oh look. You copied my link!
Sorry. No they don’t.
That’s not “rigorously defined”. It’s a bunch of weasel words and vagaries.
For example. In “factual reporting”, to get a “very high” score:
What is “consistently factual” mean? What qualifies as “a credible source”? What does “prompt” mean?
Those are all nice sounding words, but they don’t really tell you anything. Prompt could be anything from seconds to weeks. (And let’s be honest, probably varies from researcher to researcher.)
Oh they go into more detail….
A questionable source, for example:
Who defines their extreme bias? What is propaganda?
Voice of America is literally a government ran propaganda service yet they assign it high factual, least-biased and high credibility.
Sorry, but their methodology isn’t a methodology, and the only thing that’s inherently reproducible is their fact check rating. Everything else relies on what their subjective analysis.
just2look@lemm.ee 1 month ago
Consistently factual is exactly that. Both of those words mean actual things. And they go on to say that they can’t fail fact checks. And prompt corrections likely means that as a story develops, that if there were incorrect things reported, they are corrected as soon as the new information is available.
As for who defines extreme bias, it’s literally them. That is what they are saying they are doing. And they spell out what their left vs right criteria are. And how they judge it. Of course this is subjective. There isn’t really a way to judge the political spectrum without subjectivity. They do include examples in their reports about what biased language, sources, or reporting they found. Which allows you to easily judge whether you agree with it.
As for VOA, they say in the ownership portion that it is funded by the US government and that some view it as a propaganda source. They also discuss the history and purpose of it being founded. And then continue on with the factual accuracy and language analysis. You may not agree with it, but it is following their own methodology, and fully explained in the report.
Again, there isn’t anything saying you have to agree with them. It is a subjective rating. I’m not sure how much more transparent they can be though. They have spelled out how they grade, and each report provides explanations and examples that allow you to make your own judgments. Or a starting point for your own research.
If you can define a completely objective methodology to judge political bias on whatever spectrum you choose, then please do. It’s inherently subjective. And there isn’t really a way around that.
FuglyDuck@lemmy.world 1 month ago
So what constitutes “consistently factual”, then? if the ‘consistently factual’ means ‘always factual’, then the explanation of allowing ‘prompt corrections’ is unnecessary. A “correction” is different than an “update”, after all. so what rate of error is “rigorously” defined here?
Further, how do they deal with (the vast majority) of fact checkers, using qualified language like “mostly factual” or “misleading” or “out of context”. or “distorted”?
“likely…” They don’t say that. I wonder why they don’t just say that?
You’re assuming that’s what “prompt” means, but that’s… an assumption. as I said, it could be anything from seconds to weeks. I assume- i don’t know, lets just be honest here- that their language is intentional. which means it’s probably not that.
Seems like it would be a super easy thing to actually define. Like. ‘Consistently Factual’ could be “No more than X percentage of articles requiring corrections or otherwise failing a 3rd party fact check”.
So glad we agree on that.
Compare, VOA’s to Al Jazeera’s. Which, Al Jazeera is Qatar-owned. even so, It’s widely considered a reliable news source; where as, VOA was literally forbidden from being served within the US borders precisely because it was propaganda, until 2013- when it decided to open up drops to the internet specifically to “counter” Al Qaeda messaging. (aka. propaganda.)
VOA:
it should be noted that A), its so nice to know that their journalists are held to a standard. (I’m sure Al Jazeera journalists aren’t…) and b) that there’s a survey saying 84% of people that actually look at VOA is reliable. A survey conducted by… their board of governors… and the linked source is the appropriations PDF…
Compared to Al Jazeera:
now, I’m not saying Al Jazeera isn’t Qatari propaganda, it more or less is. but you see the the totally different tone here?
Now lets move onto the bias/analysis section. VOA:
Emphasis mine (also the italics just to make the headlines clear.) Now the emphasised bits is straight up bullshit. it’s government funded. It’s entire purpose- even today- is to disseminate pro-US propaganda everywhere outside the US. it’s forbiden from radio broadcasts that might reach US soil, and it’s only allowed to drop things on the interent because of a special provision specifically to counter messaging by terrorists.
Factual or not, it’s a propaganda outlet.
Al Jazeera:
now, VOA’s review is easily seen as pure spin. MBFC goes out of their way to assauge any doubt what so ever that they’re factual and not biased. nop. no sir. Now, it would be fair to say that because they literally define bias using the US discourse as the meter stick… that there is no bias. Sort of chicken and the egg, right? any how… there’s no mention of Al Jazeera’s code of ethics… and the cited failed fact checks? date to 2018, one of which falls outside the 5 year window since it was last updated- the fact check was published august of 2018, when it was updated in October of 2023. Pedantic, I know, but the 5 year window is their rule.
all it takes is a five minute scroll through VOA to see that they have the same misleading bias towards the US/US government as Al Jazeera has towards Qatar.
VOA’s was last updated in… Nov 2022.
you don’t need to define something that’s not subjective, exactly. But they need to explain what the methodology is. they’re looking for loaded words? then we need examples of what are loaded words that they’re looking for. that shouldn’t be too hard. it doesn’t even need to be exhaustive. just exhaustive enough.
Putting it on the individual articles makes it arbitrary. ask yourself… is “deadly” a loaded word? Or is it qualitative leading to understand that people actually died from the “deadly attack” rather than were just sent to the hospital in “an attack”. or that people died in a wildfire, hurricane or something else. Nobody can check every article to get a sense for their own criteria, and what they posted as a methodology is far from sufficient to the task of repeating their process. Ideally, I should be able to take their methodology article, follow it more or less step by step, and produce at least similar results. Can’t come even close.
Aurenkin@sh.itjust.works 1 month ago
There is a lot of good stuff there but it’s still opaque when it comes to bias specifically. I mean, am I missing somether here? I genuinely feel like there must be a whole section I’ve missed it something based on some of the other commenters. The bias methodology is no more a methodology than “grind up some wheat, mix some water and yeast before chucking it in the oven for a bit” is a recipe for bread. You rate 4 categories from 0 - 10 and average it, but the ratings themselves are totally subjective.
What does this even mean? If a site runs stories covering the IPCC recommendations for climate action but doesn’t run some right wing conspiracy version of how climate change is a hoax, is that biased story selection?