My point is that a LaTeX PDF can launder epistemic status. An unreviewed argument starts to look like established research merely because it adopts the visual grammar of a paper.
By the way, one of my favorite pastimes is to download the latex source for papers on arxiv and read all the commented-out stuff.
% we should make sure this theorem is actually true
“ArXiv declares independence from Cornell” (science.org)
811 points | 3 months ago | 291 comments
This isn't me siding with AI companies by the way; it's a slippery slope argument.
If Google just wanted them to exist and didn't care about profiting off of the search traffic they wouldn't partner with Mozilla.
It's also good that it doesn't gatekeep with the paywalls that you can pretty much only afford by affiliating yourself with a toll-paying institution.
Obviously, there are plenty of flaws with this system:
1. If you're associated with a brand (e.g., Google, MIT) or have a recognizable co-author (e.g., Yann LeCun), you'll get attention and citations no matter what.
2. "Vouching" can also just mean accepting someone's email request without ever having met or known them.
3. It puts the effort on the readers to decide whether each paper is valuable, and particularly scientifically valuable, for which most readers will be unequipped.
4. "Minimal standards" can be gamed by AI-generated submissions.
I'd love to see a synthesis of arXiv, open-access publishing and artifact reviews, like the following:
- Have a number of reviewers on retainer, or design a reward system similar to bug bounties. The reward mechanism probably shouldn't be based on money or allow a winner-takes-all strategy.
- Have a number of badges with respect to the quality and value of the paper. For example: validated by peers (i.e., reviewed by at least 3 peers with minimum borderline accept consensus), valuable (i.e., reviewed by at least 5 peers with a valuable indicator), etc.
- Allow vouched comments on the platform, and moderate for self-promotion, toxicity, etc. Obviously a big ask.
- Improve the "vouching" system, or add badges like "vouched by X people" or "vouched by established scientist".
Hope their new organization will implement some of these improvements.
- Organise peer feedback - Publish the work - Recognise good work, helping with both discovery and credit
That latter part especially is what allows publishers to charge the ridiculous markup that they do.
But with "modern" technology, feedback and publishing really doesn't require all that infrastructure - email and arXiv can easily be used to self-organise that. So we built a system of recognition that does not block publication, and can be used as a layer on top of arXiv and any other venue, allowing peers to vouch ("endorse") for a work.
I had even proposed and implemented an integration for arXiv Labs that got accepted, but then never merged. I should follow up on that...
You definitely should - looks like what I roughly had in mind.
Thanks for sharing!
You say it as if replication crisis doesn't exist and publish or perish is not a thing.
Removing this (often very basic) peer review doesn't somehow fix the problem. The solution lies in more thorough reviews and replication studies, not in everyone deciding for themselves.
the heel turn to unlimited for profit was only possible because of their unique structure and the fact they were already selling commercial products. arxiv is not selling anything so theres no financial incentive to take over.