Critiques of NeurIPS Conference Practices | Generated by AI
Question: What are unpopular opinions about NeurIPS?
Answer: NeurIPS (Neural Information Processing Systems), the premier conference in machine learning and AI, elicits strong reactions from the research community. While it’s celebrated for advancing the field, many participants voice frustrations over its scale, processes, and priorities. Below, I’ve compiled a selection of unpopular opinions drawn from discussions on platforms like Reddit’s r/MachineLearning and X (formerly Twitter). These highlight recurring critiques, often shared in the heat of review seasons or post-conference reflections. They’re “unpopular” because they challenge the conference’s revered status, but they’re substantiated by widespread anecdotes from reviewers, authors, and attendees.
On the Review Process
- Reviews are declining in quality due to scale and incentives, and conferences should charge submission fees to pay reviewers properly. With submission numbers ballooning (over 13,000 for NeurIPS 2024 alone), reviewers are overburdened volunteers, leading to rushed, superficial, or even AI-generated feedback. One researcher argued that without financial incentives, “the quality of reviews and discussion period will eventually go to zero,” proposing fees to fund professional reviewing.
- Inter-reviewer anonymity enables lazy or spiteful reviews. Historically, reviewers saw each other’s names, fostering accountability. Now, anonymous peers allow “short/contentless/spiteful” critiques (e.g., “why didn’t you cite my paper?”) without fear of judgment. Lifting this anonymity among reviewers—while keeping it from authors—could moderate behavior without intimidating juniors.
- Rebuttals are mostly ineffective and should be optional. Authors pour effort into responses, but reviewers rarely engage meaningfully. One suggestion: Let area chairs decide post-review if a rebuttal is even needed, sparing everyone unnecessary work.
- Banning poor reviewers from future submissions is overdue. High-profile figures like Anima Anandkumar have called out “terrible” reviews (e.g., one-sentence dismissals or major misunderstandings) and advocated barring offenders from authoring papers until they review responsibly. This echoes calls for “karma” tracking, though enforcement remains elusive.
- Simple, elegant solutions get rejected for lacking “fancy math.” Reviewers often ding strong ideas as “not novel and too simple,” pushing authors to pad papers with unnecessary theorems or complexity. One NeurIPS reviewer summed it up: “Beautiful solution of an important problem, but not novel and too simple.”
On Paper Quality and Content
- Submissions are increasingly boring, confusing, or outright fraudulent. Reviewers report LLM-generated papers (short, error-ridden, no experiments), duplicate submissions under different titles, or unreproducible work using private corporate data. One batch of five papers included just one “pretty good” entry; the rest were baffling or unethical.
- NeurIPS rewards math over real progress. The conference favors “fancy math” papers, but modern AI advances (e.g., better data curation, filtering, and feedback loops) are “unsexy” and underappreciated. This mismatch ignores what actually “moves the loss curve.”
- Too many papers are pipeline-stitching or niche benchmarks without impact. Common complaints: Off-the-shelf modules combined sub-optimally for metrics, or new benchmarks in obscure domains lacking user relevance or paradigm shifts. These feel like checkbox exercises rather than breakthroughs.
- Groupthink stifles innovation. Attendees note a “protectionist” vibe around state-of-the-art ideas, leading to marginal benchmark tweaks. The next big leap won’t come from transformers or LLMs but from bolder, riskier work.
On the Conference Experience and Accessibility
- NeurIPS feels like a corporate airport lounge, not a “meeting of minds.” At 20,000+ attendees, it’s swag-filled networking overload—recruiting, “synergies” chats, and badge-flattening exhaustion—rather than deep discourse. The real value? Serendipitous bar napkin sketches, not mega-sessions.
- It’s a flawed “ranking mechanism” that top talent skips. Viewed as an IMO/IOI-style contest sorting “smarter kids,” it burdens academics while the “smartest” join startups. Many leave uninspired by talks, with non-researchers (e.g., industry recruiters) dominating.
- Desk rejections for virtual attendance discriminate against disadvantaged researchers. A sneaky post-deadline policy change required in-person registration, screwing over those who can’t afford travel. Why penalize global participation in a field that’s already elite-heavy?
- Illegal or gimmicky posters erode the event’s prestige. Joke posters (e.g., “Generative Boba” or startup ads mimicking real ones) exploit coveted spots, turning hard-earned acceptances into sideshows. Is it joy-sparking or just clout-chasing?
These opinions reflect a conference at a crossroads: NeurIPS drives AI forward but strains under its success. Many suggest reforms like splitting into tracks, better incentives, or emphasizing reproducibility. If you’re submitting or attending, brace for the chaos—it’s part of the charm (or curse).
References:
- NeurIPS Paper Reviews Released, Controversies Resurface
- r/MachineLearning: [D] NeurIPS 2024 Paper Reviews
- r/MachineLearning: [D] NeurIPS 2025 Reviews
- X Post by @jxmnop on reviewing awful papers
- X Post by @DBahdanau on NeurIPS as a ranking mechanism
- X Post by @tsunghan_wu on unexciting paper types
- X Post by @rao2z on inter-reviewer anonymity
- X Post by @NeelNanda5 on virtual attendance policy
- X Post by @arrtnem on conference overload