To give you a little history. I was seeking to create a system that would evaluate scientific papers so that I and others could better understand what was actually being said. So, I created Design Biology, but that was too narrow, so I created FEP, which stands for Forensic Evaluation Protocol. Then, of course, I began searching for papers to evaluate. Lo and behold, a paper that appears to be attempting the same tasks I was doing described a tool they were building. That is when I ran a full FEP evaluation, both negative and positive on the paper titled “Initiating 'discourse analysis' as a tool to differentiate between science and pseudoscience: Another valuable tool to advance objectivity and rigor in science (IJISRT, June 2024).[1] While I disagreed with parts of its framing, I saw real promise in the tool itself.
So I integrated discourse analysis into FEP as a Language and Argument Forensics module. I treat it as an early warning system, not a verdict machine.
Where it fits inside FEP
I place it in two spots.
First, right after the Claim Map.
This catches claim inflation early, before I waste time auditing the wrong target.
Second, right after the Evidence Ledger.
This checks the match between rhetoric and reality. Strong words with weak evidence show up fast. Careful words with strong evidence also show up fast.
What it adds is that FEP often needs
FEP is strong at data, methods, incentives, and falsifiers. It can miss the soft layer. That soft layer is the writing and argument moves that hide weak reasoning in plain sight.
Discourse analysis provides the evaluator with a clear vocabulary for common failure patterns.
1. Definitional drift
Key terms like “science,” “pseudoscience,” “objectivity,” and “rigor” get used loosely. Then they get stretched to cover new claims. When definitions shift, the entire evaluation shifts with them. That is where we step and push back.
2. Unfalsifiable framing
Some statements never name a failure condition. A method that cannot fail will not self-correct in practice.
3. Fallacy markers and “weasel talk.”
I look for straw men, non sequiturs, appeals to authority, cherry-pick cues, and vague blame language. These patterns often signal that the argument is running ahead of the evidence.
4. Scope creep
Text begins as “a tool to assist review.” Later, it becomes “a tool to distinguish science from pseudoscience.” That shift matters because it changes the claim type without changing the evidence burden.
5. Ideology cues
Power language, as identified in Critical Discourse Analysis, can expose real bias. It can also serve as a tool for punishing unpopular ideas. That is why I treat ideological cues as text signals only, not as mind-reading.
How do I add it without weakening FEP?
A language module can strengthen FEP or contaminate it. I keep it scientific by using three guardrails.
A. Separate three categories
- Bad writing
- Bad reasoning
- Bad science (methods and evidence)
Discourse analysis mainly sees category 1 and parts of category 2. FEP still has to test category 3 with evidence.
B. Turn discourse checks into scored risk signals, not labels
I do not stamp work as “pseudoscience.”
I write what I can actually defend from the text, such as:
High rhetoric-to-evidence mismatch.
Definitional instability.
No stated falsifier.
That keeps the evaluation anchored to observable features.
C. Require external validation for the discourse score
If the discourse score predicts nothing outside the document, it stays a style critique.
If it predicts real-world outcomes, such as replication failure, chronic non-correction, or recurring retractions, it becomes a real diagnostic signal.
The clean embed: a new section in every FEP report
I added one section to the standard FEP template.
Section: Discourse & Argument Quality (DAQ)
- Claim inflation check
- Definitions check
- Fallacy marker check
- Language falsifiability check
- Rhetoric-to-evidence mismatch check
- Bias cue check (text-only, no mind-reading)
- Notes for reviewer training and inter-rater agreement
This makes FEP harder to game. Weak work can no longer hide behind polished prose, moral language, or confident tone.
Why this matters
Most readers do not audit methods. They audit the vibe. Bad actors know that. Even honest researchers fall into it under pressure to publish, persuade, or win a debate.
DAQ gives me (the evaluator) a disciplined way to say, “Stop. The language is outrunning the support.” Then FEP does what it does best. It tests the actual science. So, please drop me a line, tell me what you think. Do you have any ideas on how we can evaluate and check the thousands of scientific papers that are pushed out each year? I would love to hear from you.
[1] Volume 9 2024_issue 6 June | International Journal of Innovative Science and Research Technology. https://www.ijisrt.com/Volume-9-2024_issue-6-june