--- type: "firecrawl-provider" description: "OpenReview (openreview.net): full-text search over public submissions (title, abstract, authors, venue, PDF/HTML links, BibTeX) and over public reviews, meta-reviews, decisions and comments, the catalog of venues hosted on OpenReview and one venue's public configuration, all through the anonymous, stateless api2.openreview.net (v2) and api.openreview.net (v1) JSON endpoints. Browsing a venue's full submission list, fetching one paper or its review thread by id, and PDF download sit behind OpenReview's first-party Cloudflare Turnstile gate and are not offered." use_when: "OpenReview (openreview.net): full-text search over public submissions (title, abstract, authors, venue, PDF/HTML links, BibTeX) and over public reviews, meta-reviews, decisions and comments, the catalog of venues hosted on OpenReview and one venue's public configuration, all through the anonymous, stateless api2.openreview.net (v2) and api.openreview.net (v1) JSON endpoints. Browsing a venue's full submission list, fetching one paper or its review thread by id, and PDF download sit behind OpenReview's first-party Cloudflare Turnstile gate and are not offered." categories: "AI models" capabilities: 4 credits_per_call: 5 --- # OpenReview on Firecrawl Alexandria OpenReview (openreview.net): full-text search over public submissions (title, abstract, authors, venue, PDF/HTML links, BibTeX) and over public reviews, meta-reviews, decisions and comments, the catalog of venues hosted on OpenReview and one venue's public configuration, all through the anonymous, stateless api2.openreview.net (v2) and api.openreview.net (v1) JSON endpoints. Browsing a venue's full submission list, fetching one paper or its review thread by id, and PDF download sit behind OpenReview's first-party Cloudflare Turnstile gate and are not offered. - Categories: AI models - Category index: [AI models category](https://firecrawl.dev/alexandria/agents/categories/ai-models) - Provider key: `openreview-net` - Access: Firecrawl credits - Cost: 5 credits per call ## More - [Human guide](https://firecrawl.dev/app/alexandria/openreview-net) - [OpenAPI spec](https://firecrawl.dev/alexandria/agents/providers/openreview-net/openapi.json) ## Capabilities - [Get venue](https://firecrawl.dev/alexandria/agents/providers/openreview-net/papers/get_venue): One venue group on OpenReview by id or page URL, through GET /groups?id= on api2.openreview.net: title, subtitle, website, location, dates, contact, parent and domain, the submission invitation and the venue ids it files accepted / withdrawn / desk-rejected / rejected submissions under, whether submissions are public, and the names the venue gives reviews, meta-reviews and decisions. Fields the venue has not configured are null (journals such as TMLR expose a smaller configuration than conferences). - [List venues](https://firecrawl.dev/alexandria/agents/providers/openreview-net/papers/list_venues): The venues hosted on OpenReview, from the `venues` (every venue, about 4700 ids) or `active_venues` (currently active, about 1100 ids) root groups on api2.openreview.net, optionally filtered by a case-insensitive substring of the id, paginated client-side. Each entry is the venue group id (`ICLR.cc/2025/Conference`, `TMLR`, `auai.org/UAI/2022/Conference`) with its OpenReview page URL. - [Search replies](https://firecrawl.dev/alexandria/agents/providers/openreview-net/papers/search_replies): Full-text search over public reply notes on OpenReview: official reviews, meta-reviews, decisions, author responses and comments, optionally scoped to one venue group, through GET /notes/search?source=reply on api2.openreview.net. Each reply carries its kind (the invitation suffix such as `Official_Review`, `Official_Comment`, `Meta_Review`, `Decision`), the forum (paper) id and title, the reply-to id, signatures, the common review fields (summary, strengths, weaknesses, questions, rating, confidence, soundness, presentation, contribution, comment) and the complete form content. - [Search submissions](https://firecrawl.dev/alexandria/agents/providers/openreview-net/papers/search_submissions): Full-text search over public OpenReview submissions (papers), optionally scoped to one venue group, through GET /notes/search?source=forum. Returns one page of submissions with title, abstract, authors, author profile ids, venue label and venue id (which distinguishes accepted, withdrawn, rejected and desk-rejected submissions), keywords, TL;DR, PDF and HTML links, license, BibTeX and timestamps, plus the total hit count (the v2 index caps it at 10000) and the next offset. `api: v1` queries the legacy api.openreview.net index (venues before 2023 and dblp records; 5 requests per minute per IP) instead of the default v2 index. ## 1. Choose this provider when OpenReview (openreview.net): full-text search over public submissions (title, abstract, authors, venue, PDF/HTML links, BibTeX) and over public reviews, meta-reviews, decisions and comments, the catalog of venues hosted on OpenReview and one venue's public configuration, all through the anonymous, stateless api2.openreview.net (v2) and api.openreview.net (v1) JSON endpoints. Browsing a venue's full submission list, fetching one paper or its review thread by id, and PDF download sit behind OpenReview's first-party Cloudflare Turnstile gate and are not offered. ## 2. Minimal request Call `POST https://api.firecrawl.dev/v2/scrape` with `{ alexandria: { provider, capability, options } }`. For a batch, send `{ alexandria: [...] }` with up to 10 calls. ```json { "provider": "openreview-net", "capability": "papers/get_venue", "options": { "venue_id": "ICLR.cc/2025/Conference" } } ``` ## 3. Add provider options Use only the options needed for the task: - `url` (string): A venue page URL, https://openreview.net/group?id={venue_id}. Exactly one of `venue_id` or `url`. Example: `venue_id` - `venue_id` (string): Venue group id, e.g. `ICLR.cc/2025/Conference`, `NeurIPS.cc/2024/Conference`, `TMLR`. Example: `ICLR.cc/2025/Conference` ## 4. Request through your preferred interface ### JavaScript ```javascript const result = await firecrawl.scrape({ alexandria: { provider: "openreview-net", capability: "papers/get_venue", options: { venue_id: "ICLR.cc/2025/Conference", }, }, }); ``` ### Python ```python result = firecrawl.scrape_alexandria({ "provider": "openreview-net", "capability": "papers/get_venue", "options": { "venue_id": "ICLR.cc/2025/Conference" } }) ``` ### cURL ```sh curl https://api.firecrawl.dev/v2/scrape \ -H "Authorization: Bearer $FIRECRAWL_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "alexandria": { "provider": "openreview-net", "capability": "papers/get_venue", "options": { "venue_id": "ICLR.cc/2025/Conference" } } }' ``` ### CLI ```sh firecrawl scrape 'openreview-net/papers/get_venue' \ --options '{"venue_id":"ICLR.cc/2025/Conference"}' ``` ### MCP Call the FCX MCP retrieve tool with this object: ```json { "provider": "openreview-net", "capability": "papers/get_venue", "options": { "venue_id": "ICLR.cc/2025/Conference" } } ``` Ask for only the returned fields needed by the task. ## 5. Full request shape ```json { "provider": "openreview-net", "capability": "papers/get_venue", "options": { "venue_id": "ICLR.cc/2025/Conference" } } ``` ## 6. Response data The response includes `success`, `provider`, `capability`, `creditsCost` and `data`. This example shows the provider payload in `data`: ```json { "accept_decision_options": [ "Accept (Oral)", "Accept (Spotlight)", "Accept (Poster)", "Accept (conditional oral)", "Accept (conditional spotlight)", "Accept (conditional poster)" ], "area_chairs_name": "Area_Chairs", "contact": "program-chairs@iclr.cc", "created_at_ms": 1720731165376, "dates": "Submission Start: Sep 13 2024 12:00AM UTC-0, Abstract Registration: Sep 28 2024 11:59AM UTC-0, Submission Deadline: Oct 02 2024 11:59AM UTC-0", "decision_name": "Decision", "desk_rejected_venue_id": "ICLR.cc/2025/Conference/Desk_Rejected_Submission", "domain": "ICLR.cc/2025/Conference", "host": "ICLR.cc", "id": "ICLR.cc/2025/Conference", "instructions": "", "location": "Singapore", "meta_review_name": "Meta_Review", "modified_at_ms": 1748894759251, "observed_at_ms": 1790993626012, "parent": "ICLR.cc/2025", "public_desk_rejected_submissions": true, "public_submissions": true, "public_withdrawn_submissions": true, "rejected_venue_id": "ICLR.cc/2025/Conference/Rejected_Submission", "review_name": "Official_Review", "reviewers_name": "Reviewers", "source_url": "https://api2.openreview.net/groups?id=ICLR.cc%2F2025%2FConference", "start_date": "Apr 24 2025", "submission_invitation": "ICLR.cc/2025/Conference/-/Submission", "submission_venue_id": "ICLR.cc/2025/Conference/Submission", "subtitle": "ICLR 2025", "title": "The Thirteenth International Conference on Learning Representations", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FConference", "website": "https://iclr.cc/Conferences/2025", "withdrawn_venue_id": "ICLR.cc/2025/Conference/Withdrawn_Submission" } ``` ## API reference-derived contract The following capability contract is generated from the same normalized Alexandria API reference exposed in the API spec. ### Get venue - Capability: `papers/get_venue` - Description: One venue group on OpenReview by id or page URL, through GET /groups?id= on api2.openreview.net: title, subtitle, website, location, dates, contact, parent and domain, the submission invitation and the venue ids it files accepted / withdrawn / desk-rejected / rejected submissions under, whether submissions are public, and the names the venue gives reviews, meta-reviews and decisions. Fields the venue has not configured are null (journals such as TMLR expose a smaller configuration than conferences). - Instructions: Resolve a venue id from list_venues or a `venue_id` from search results into its name, dates and public-submission policy, or check which venue ids to filter search results by. An unknown group id is `not_found`. - Cost: 5 credits per call - Capability file: [Get venue](https://firecrawl.dev/alexandria/agents/providers/openreview-net/papers/get_venue) Accepted options: - `url` (string): A venue page URL, https://openreview.net/group?id={venue_id}. Exactly one of `venue_id` or `url`. Example: `venue_id` - `venue_id` (string): Venue group id, e.g. `ICLR.cc/2025/Conference`, `NeurIPS.cc/2024/Conference`, `TMLR`. Example: `ICLR.cc/2025/Conference` Response schema example: ```json { "accept_decision_options": [ "Accept (Oral)", "Accept (Spotlight)", "Accept (Poster)", "Accept (conditional oral)", "Accept (conditional spotlight)", "Accept (conditional poster)" ], "area_chairs_name": "Area_Chairs", "contact": "program-chairs@iclr.cc", "created_at_ms": 1720731165376, "dates": "Submission Start: Sep 13 2024 12:00AM UTC-0, Abstract Registration: Sep 28 2024 11:59AM UTC-0, Submission Deadline: Oct 02 2024 11:59AM UTC-0", "decision_name": "Decision", "desk_rejected_venue_id": "ICLR.cc/2025/Conference/Desk_Rejected_Submission", "domain": "ICLR.cc/2025/Conference", "host": "ICLR.cc", "id": "ICLR.cc/2025/Conference", "instructions": "", "location": "Singapore", "meta_review_name": "Meta_Review", "modified_at_ms": 1748894759251, "observed_at_ms": 1790993626012, "parent": "ICLR.cc/2025", "public_desk_rejected_submissions": true, "public_submissions": true, "public_withdrawn_submissions": true, "rejected_venue_id": "ICLR.cc/2025/Conference/Rejected_Submission", "review_name": "Official_Review", "reviewers_name": "Reviewers", "source_url": "https://api2.openreview.net/groups?id=ICLR.cc%2F2025%2FConference", "start_date": "Apr 24 2025", "submission_invitation": "ICLR.cc/2025/Conference/-/Submission", "submission_venue_id": "ICLR.cc/2025/Conference/Submission", "subtitle": "ICLR 2025", "title": "The Thirteenth International Conference on Learning Representations", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FConference", "website": "https://iclr.cc/Conferences/2025", "withdrawn_venue_id": "ICLR.cc/2025/Conference/Withdrawn_Submission" } ``` ### List venues - Capability: `papers/list_venues` - Description: The venues hosted on OpenReview, from the `venues` (every venue, about 4700 ids) or `active_venues` (currently active, about 1100 ids) root groups on api2.openreview.net, optionally filtered by a case-insensitive substring of the id, paginated client-side. Each entry is the venue group id (`ICLR.cc/2025/Conference`, `TMLR`, `auai.org/UAI/2022/Conference`) with its OpenReview page URL. - Instructions: Discover the venue group id to pass as `venue` to the search functions or to get_venue, e.g. every ICLR venue with filter `ICLR.cc`. Titles and dates come from get_venue. - Cost: 5 credits per call - Capability file: [List venues](https://firecrawl.dev/alexandria/agents/providers/openreview-net/papers/list_venues) Accepted options: - `filter` (string): Case-insensitive substring the venue id must contain, e.g. `ICLR.cc`, `2025`, `Workshop`. Example: `ICLR.cc` - `limit` (number): Entries per page (default 100). Example: `10` - `offset` (number): Zero-based offset into the filtered list; pass the previous page's `next_offset` to continue. Example: `10` - `scope` (string): `active` (default): venues OpenReview lists as currently active; `all`: every venue group ever hosted. Example: `active` Response schema example: ```json { "count": 41, "filter": "ICLR.cc/2025", "limit": 50, "next_offset": null, "observed_at_ms": 1790993622062, "offset": 0, "scope": "all", "source_url": "https://api2.openreview.net/groups?id=venues", "total": 41, "venues": [ { "id": "ICLR.cc/2025/BlogPosts", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FBlogPosts" }, { "id": "ICLR.cc/2025/Workshop/World_Models", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FWorld_Models" }, { "id": "ICLR.cc/2025/Workshop/FPI", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FFPI" }, { "id": "ICLR.cc/2025/Workshop/ICBINB", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FICBINB" }, { "id": "ICLR.cc/2025/Workshop/GEM", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FGEM" }, { "id": "ICLR.cc/2025/Workshop/HAIC", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FHAIC" }, { "id": "ICLR.cc/2025/Workshop/LLM_Reason_and_Plan", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FLLM_Reason_and_Plan" }, { "id": "ICLR.cc/2025/Workshop/SCI-FM", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FSCI-FM" }, { "id": "ICLR.cc/2025/Workshop/AI4MAT", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FAI4MAT" }, { "id": "ICLR.cc/2025/Workshop/LMRL", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FLMRL" }, { "id": "ICLR.cc/2025/Workshop/Bi-Align", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FBi-Align" }, { "id": "ICLR.cc/2025/Workshop/Financial_AI", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FFinancial_AI" }, { "id": "ICLR.cc/2025/Workshop/VerifAI", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FVerifAI" }, { "id": "ICLR.cc/2025/Workshop/SSI-FM", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FSSI-FM" }, { "id": "ICLR.cc/2025/Workshop/MLDPR", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FMLDPR" }, { "id": "ICLR.cc/2025/Workshop/Re-Align", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FRe-Align" }, { "id": "ICLR.cc/2025/Workshop/MCDC", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FMCDC" }, { "id": "ICLR.cc/2025/Workshop/FM-Wild", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FFM-Wild" }, { "id": "ICLR.cc/2025/Workshop/SCSL", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FSCSL" }, { "id": "ICLR.cc/2025/Workshop/DL4C", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FDL4C" }, { "id": "ICLR.cc/2025/Workshop/SCOPE", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FSCOPE" }, { "id": "ICLR.cc/2025/Workshop/NFAM", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FNFAM" }, { "id": "ICLR.cc/2025/Workshop/DeLTa", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FDeLTa" }, { "id": "ICLR.cc/2025/Workshop/AI4NA", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FAI4NA" }, { "id": "ICLR.cc/2025/Workshop/QUESTION", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FQUESTION" }, { "id": "ICLR.cc/2025/Workshop/SLLM", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FSLLM" }, { "id": "ICLR.cc/2025/Workshop/Data_Problems", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FData_Problems" }, { "id": "ICLR.cc/2025/Workshop/MLMP", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FMLMP" }, { "id": "ICLR.cc/2025/Workshop/WSL", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FWSL" }, { "id": "ICLR.cc/2025/Workshop/BuildingTrust", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FBuildingTrust" }, { "id": "ICLR.cc/2025/Workshop/AgenticAI", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FAgenticAI" }, { "id": "ICLR.cc/2025/Workshop/WMARK", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FWMARK" }, { "id": "ICLR.cc/2025/Workshop/AI4CHL", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FAI4CHL" }, { "id": "ICLR.cc/2025/Workshop/XAI4Science", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FXAI4Science" }, { "id": "ICLR.cc/2025/Workshop/MLGenX", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FMLGenX" }, { "id": "ICLR.cc/2025/Workshop/WRL", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FWRL" }, { "id": "ICLR.cc/2025/Workshop/SynthData", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FSynthData" }, { "id": "ICLR.cc/2025/Workshop/EmbodiedAI", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FEmbodiedAI" }, { "id": "ICLR.cc/2025/Workshop/MLMP-IRT", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop%2FMLMP-IRT" }, { "id": "ICLR.cc/2025/Conference", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FConference" }, { "id": "ICLR.cc/2025/Workshop_Proposals", "url": "https://openreview.net/group?id=ICLR.cc%2F2025%2FWorkshop_Proposals" } ] } ``` ### Search replies - Capability: `papers/search_replies` - Description: Full-text search over public reply notes on OpenReview: official reviews, meta-reviews, decisions, author responses and comments, optionally scoped to one venue group, through GET /notes/search?source=reply on api2.openreview.net. Each reply carries its kind (the invitation suffix such as `Official_Review`, `Official_Comment`, `Meta_Review`, `Decision`), the forum (paper) id and title, the reply-to id, signatures, the common review fields (summary, strengths, weaknesses, questions, rating, confidence, soundness, presentation, contribution, comment) and the complete form content. - Instructions: Find public reviews or discussion that mention a topic, or read how a venue's reviewers rated papers on a subject. Review visibility is venue policy (ICLR publishes reviews; many venues keep them private). Replies cannot be filtered to one paper on this surface: the per-forum thread sits behind the Turnstile gate; group results client-side by `forum` instead. - Cost: 5 credits per call - Capability file: [Search replies](https://firecrawl.dev/alexandria/agents/providers/openreview-net/papers/search_replies) Accepted options: - `limit` (number): Results per page, 1-100 (default 25). Example: `10` - `offset` (number): Zero-based offset of the first result; pass the previous page's `next_offset` to continue. Example: `10` - `query` (string, required): Search terms matched against the reply content (full-text). Must contain a non-whitespace character. Example: `` - `venue` (string): Venue group id to scope the search to, e.g. `ICLR.cc/2025/Conference`. Omit or pass `all` for every venue. Example: `ICLR.cc/2025/Conference` Response schema example: ```json { "count": 2, "limit": 2, "next_offset": 2, "observed_at_ms": 1790993618001, "offset": 0, "query": "diffusion", "replies": [ { "comment": null, "confidence": 5, "content": { "code_of_conduct": "Yes", "confidence": 5, "contribution": 2, "flag_for_ethics_review": [ "No ethics review needed." ], "presentation": 2, "questions": "My questions and suggestions are mostly covers in the weaknesses outlined above.", "rating": 3, "soundness": 2, "strengths": "The ablation study design is both interesting and essential to clearing a number of doubts about this paper, although the results are not exactly convincing. The doubts include: 1) Why not directly attack the probing classifier? 2) Why use DiffusionSeq?, etc.", "summary": "In this paper, the authors proposed DiffusionAttacker as a while-box LLM jailbreaking method, which attacks a LLM by attacking a harmfulness probing classifier trained on its last layer hidden states instead, based on the observation that the jailbreaking prompts produced by some existing jailbreaking methods are often no longer sparable from non-harmful inputs by the classifier. In specific, DiffusionAttacker employs DiffusionSeq models to rewrite malicious requests through denoising to 1) preserve the semantic content, while 2) cheating the probing classifier by end-to-end training (with the help of Gumbel SoftMax for differentiability) with the victim LLM (frozen). DiffusionAttacker is tested on the AdvBench dataset against Llama 3 8b, Mistral 7b, Alpaca 7b (and Vicuna 1.5 7b) models and showed improvement in ASR (by prefix- and GPT-judge), PPL and self-bleu over existing attacks including GCG, AutoDan and ColdAttack. The transfer study showed that jailbreaking prompts generated by DiffusionAttacker are also more likely to jailbreak other models. The ablation study further investigated the impact of each component of DiffusionAttack.", "weaknesses": "1. Inadequate Baseline Selection: While the likes of GCG and AutoDAN are of clear resemblance with DiffusionAttacker method-wise (white-box, proxy-target, guidance from gradient), they are not among the most powerful jailbreaking methods. A number of black-box attacks like [1] and [2] can also jailbreak the white-box models while being more fluent, efficient and effective.\n2. Questionable Metric: ASR computed by prefix match is outdated and knowingly unable to reflect the true jailbreaking effectiveness. The GPT-judge used is also not a widely-used or \"strong\" one, in that it is only employing GPT-4o to decide the harmfulness without detailed criteria, judgement on a scale instead of binary, or consideration of content relatedness to the malicious request, etc. as identified by e.g. [2] and [3] as crucial properties for a GPT-judge to evaluate jailbreaking attacks more accurately. There are not adequate reasons to incorporate PPL and Self-Bleu scores as the major metrics. Why is fluency of the jailbreaking prompt an important characteristic to have? There does exist defensive mechanisms that uses PPL to filter suspectable inputs, but they haven't been widely used ever. For instance, in the interactive with LLMs, rare token sequences have good reasons to appear, e.g. ASCII arts, ciphertexts, etc. Bleu-based diversity is also not a guarantee that a jailbreaking method is hard to defend against.\n3. Non-prominent Performance: The ASRs (GPT-based), while higher than the few baselines, are not actually high from today's point of view given that a number of black-box attacks have surpassed them. Many of them also make use of the idea about turning the prompt less harmful by the look to generate jailbreaking prompts. In a sense, DiffusionAttacker is only resorting to the internal hidden states of the victim LLMs which black-box attacks refrain from to do a similar thing while not receiving any benefit in terms of fluency, effectiveness or efficiency. The transfer attack ASRs are also very low to be of practical use.\n4. Questionable Design Choices: Many design choices in the paper are not well-justified, e.g. 1) Why choosing the last-layer hidden states for the classifier, when a number of works has identified middle layers as more effective at probing? 2) Why using DiffusionSeq when there are alternatives like encoder-decoder transfomers and LSTMs? What is special about diffusion models that make it fit for this task? 3) \n5. Inadequate Experiments: Being an attack that stems from representation engineering [4], clearly [5] is a better baseline as it is finetuned specifically to direct harmful representations to benign ones. A number of jailbreaking methods have failed at attack it while exile otherwise, so if DiffusionAttack manages to break [5] due to its similar concepts, then the relatively inadequate performance might be justified a bit as it at least finds some unique situations where it is the only successful attack. Additionally, even in the transfer experiment, only white-box models are used. It is important to see how the prompts generated by DiffusionAttack attacks GPTs, Geminis and Claudes.\n6. Poor Demonstration: The paper provides no examples of the jailbreaking outcomes. There is also no graph to illustrate the change in classifier decision after the attack to validate the hypothesize which motivates this paper.\n7. Questionable Ablation: The advantage of DiffusionAttack over the other design choices is not significant except for fluency which is obviously going to be the case.\n\n[1] Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues\n[2] WordGame: Efficient & Effective LLM Jailbreak via Simultaneous Obfuscation in Query and Response.\n[3] A StrongREJECT for Empty Jailbreaks\n[4] Representation Engineering: A Top-Down Approach to AI Transparency\n[5] Improving Alignment and Robustness with Circuit Breakers" }, "contribution": 2, "created_at_ms": 1730659429985, "domain": "ICLR.cc/2025/Conference", "forum": "u08UxVNdIo", "forum_url": "https://openreview.net/forum?id=u08UxVNdIo¬eId=MjhJoKn4g3", "id": "MjhJoKn4g3", "invitation": "ICLR.cc/2025/Conference/Submission3818/-/Official_Review", "kind": "Official_Review", "modified_at_ms": 1731427882305, "number": 4, "paper_title": "Diffusion Attacker: Diffusion-Driven Prompt Manipulation for LLM Jailbreak", "paper_venue": "ICLR 2025 Conference Withdrawn Submission", "paper_venue_id": "ICLR.cc/2025/Conference/Withdrawn_Submission", "presentation": 2, "questions": "My questions and suggestions are mostly covers in the weaknesses outlined above.", "rating": 3, "readers": [ "everyone" ], "reply_to": "u08UxVNdIo", "signatures": [ "ICLR.cc/2025/Conference/Submission3818/Reviewer_MRHd" ], "soundness": 2, "strengths": "The ablation study design is both interesting and essential to clearing a number of doubts about this paper, although the results are not exactly convincing. The doubts include: 1) Why not directly attack the probing classifier? 2) Why use DiffusionSeq?, etc.", "summary": "In this paper, the authors proposed DiffusionAttacker as a while-box LLM jailbreaking method, which attacks a LLM by attacking a harmfulness probing classifier trained on its last layer hidden states instead, based on the observation that the jailbreaking prompts produced by some existing jailbreaking methods are often no longer sparable from non-harmful inputs by the classifier. In specific, DiffusionAttacker employs DiffusionSeq models to rewrite malicious requests through denoising to 1) preserve the semantic content, while 2) cheating the probing classifier by end-to-end training (with the help of Gumbel SoftMax for differentiability) with the victim LLM (frozen). DiffusionAttacker is tested on the AdvBench dataset against Llama 3 8b, Mistral 7b, Alpaca 7b (and Vicuna 1.5 7b) models and showed improvement in ASR (by prefix- and GPT-judge), PPL and self-bleu over existing attacks including GCG, AutoDan and ColdAttack. The transfer study showed that jailbreaking prompts generated by DiffusionAttacker are also more likely to jailbreak other models. The ablation study further investigated the impact of each component of DiffusionAttack.", "title": null, "weaknesses": "1. Inadequate Baseline Selection: While the likes of GCG and AutoDAN are of clear resemblance with DiffusionAttacker method-wise (white-box, proxy-target, guidance from gradient), they are not among the most powerful jailbreaking methods. A number of black-box attacks like [1] and [2] can also jailbreak the white-box models while being more fluent, efficient and effective.\n2. Questionable Metric: ASR computed by prefix match is outdated and knowingly unable to reflect the true jailbreaking effectiveness. The GPT-judge used is also not a widely-used or \"strong\" one, in that it is only employing GPT-4o to decide the harmfulness without detailed criteria, judgement on a scale instead of binary, or consideration of content relatedness to the malicious request, etc. as identified by e.g. [2] and [3] as crucial properties for a GPT-judge to evaluate jailbreaking attacks more accurately. There are not adequate reasons to incorporate PPL and Self-Bleu scores as the major metrics. Why is fluency of the jailbreaking prompt an important characteristic to have? There does exist defensive mechanisms that uses PPL to filter suspectable inputs, but they haven't been widely used ever. For instance, in the interactive with LLMs, rare token sequences have good reasons to appear, e.g. ASCII arts, ciphertexts, etc. Bleu-based diversity is also not a guarantee that a jailbreaking method is hard to defend against.\n3. Non-prominent Performance: The ASRs (GPT-based), while higher than the few baselines, are not actually high from today's point of view given that a number of black-box attacks have surpassed them. Many of them also make use of the idea about turning the prompt less harmful by the look to generate jailbreaking prompts. In a sense, DiffusionAttacker is only resorting to the internal hidden states of the victim LLMs which black-box attacks refrain from to do a similar thing while not receiving any benefit in terms of fluency, effectiveness or efficiency. The transfer attack ASRs are also very low to be of practical use.\n4. Questionable Design Choices: Many design choices in the paper are not well-justified, e.g. 1) Why choosing the last-layer hidden states for the classifier, when a number of works has identified middle layers as more effective at probing? 2) Why using DiffusionSeq when there are alternatives like encoder-decoder transfomers and LSTMs? What is special about diffusion models that make it fit for this task? 3) \n5. Inadequate Experiments: Being an attack that stems from representation engineering [4], clearly [5] is a better baseline as it is finetuned specifically to direct harmful representations to benign ones. A number of jailbreaking methods have failed at attack it while exile otherwise, so if DiffusionAttack manages to break [5] due to its similar concepts, then the relatively inadequate performance might be justified a bit as it at least finds some unique situations where it is the only successful attack. Additionally, even in the transfer experiment, only white-box models are used. It is important to see how the prompts generated by DiffusionAttack attacks GPTs, Geminis and Claudes.\n6. Poor Demonstration: The paper provides no examples of the jailbreaking outcomes. There is also no graph to illustrate the change in classifier decision after the attack to validate the hypothesize which motivates this paper.\n7. Questionable Ablation: The advantage of DiffusionAttack over the other design choices is not significant except for fluency which is obviously going to be the case.\n\n[1] Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues\n[2] WordGame: Efficient & Effective LLM Jailbreak via Simultaneous Obfuscation in Query and Response.\n[3] A StrongREJECT for Empty Jailbreaks\n[4] Representation Engineering: A Top-Down Approach to AI Transparency\n[5] Improving Alignment and Robustness with Circuit Breakers" }, { "comment": null, "confidence": 3, "content": { "code_of_conduct": "Yes", "confidence": 3, "contribution": 2, "flag_for_ethics_review": [ "No ethics review needed." ], "presentation": 3, "questions": "I have several questions about this work.\n\n1, How to decide the scaling factors? Since the intervals are [1000,5000], [100,500], [100,500], [100,500], and the values seesm to be integer, then the whole factor space equals 4000 * 400 * 400 * 400, which is quite huge. And the authors present one setting for NASBench201 and other experiments, respectively, so I am wondering whether there is some method or strategy to choose such factors? \n\n2, This work extends the basic motivation of DiffusionNAG, which is rather good and natural. Such extension include three more factors, including number of parameters, number of MACs, and the inference latency. But I am curious that, how about the performance of POMONAG if just considering adding one factor? \n\n3, From one factor, say, accuracy, to three more factors seems strenghening the proposed POMONAG, but my question is, the working mechanism of DiffusionNAG and POMONAG the same different? Although the two diffusion processes consider different factors, which is the obvious difference, but the analysis or discussion is important to interpret this issue.", "rating": 5, "soundness": 3, "strengths": "This paper introduces the ParetoOptimal Many-Objective Neural Architecture Generator (POMONAG), extending DiffusionNAG through a many-objective diffusion process. POMONAG simultaneously considers accuracy, the number of parameters, multiply-accumulate operations (MACs), and inference latency. The experiments validate the performance of the proposed model.", "summary": "This paper is a direct extension based on DiffusionNAG, which can deal with multi-objective optimization in NAS. These objectives include accuracy, the number of parameters, multiply-accumulate operations (MACs), and inference latency. This motivation is good and natural, and the authors expressed their work clearly, from the motivation to the experiments results. Some details need to be clarified.", "weaknesses": "1, The multi-objective optimization problem formulation in this work can be given first, which then can be solved by the proposed weighted factors in the reverse diffusion process. But maybe the authors can consider other ways to sovle this. For example, using four single reverse diffusion process each targeting one factor, as DiffusionNAG did, then using multi-objective optimization for further trade-off may also work well.\n\n2, The theoretical analysis should be strehghen. One objective to many objective is a breakthrough, but such process needs more analysis or discussion. Current work lacks such in-depth thinking. \n\n3, Several predictors are needed in this work, but the detailed information these predictors are missing." }, "contribution": 2, "created_at_ms": 1730597839913, "domain": "ICLR.cc/2025/Conference", "forum": "4y4t7yOvJO", "forum_url": "https://openreview.net/forum?id=4y4t7yOvJO¬eId=6WNnptOc5w", "id": "6WNnptOc5w", "invitation": "ICLR.cc/2025/Conference/Submission4704/-/Official_Review", "kind": "Official_Review", "modified_at_ms": 1731428201981, "number": 3, "paper_title": "POMONAG: Pareto-Optimal Many-Objective Neural Architecture Generator", "paper_venue": "Submitted to ICLR 2025", "paper_venue_id": "ICLR.cc/2025/Conference/Rejected_Submission", "presentation": 3, "questions": "I have several questions about this work.\n\n1, How to decide the scaling factors? Since the intervals are [1000,5000], [100,500], [100,500], [100,500], and the values seesm to be integer, then the whole factor space equals 4000 * 400 * 400 * 400, which is quite huge. And the authors present one setting for NASBench201 and other experiments, respectively, so I am wondering whether there is some method or strategy to choose such factors? \n\n2, This work extends the basic motivation of DiffusionNAG, which is rather good and natural. Such extension include three more factors, including number of parameters, number of MACs, and the inference latency. But I am curious that, how about the performance of POMONAG if just considering adding one factor? \n\n3, From one factor, say, accuracy, to three more factors seems strenghening the proposed POMONAG, but my question is, the working mechanism of DiffusionNAG and POMONAG the same different? Although the two diffusion processes consider different factors, which is the obvious difference, but the analysis or discussion is important to interpret this issue.", "rating": 5, "readers": [ "everyone" ], "reply_to": "4y4t7yOvJO", "signatures": [ "ICLR.cc/2025/Conference/Submission4704/Reviewer_s4FH" ], "soundness": 3, "strengths": "This paper introduces the ParetoOptimal Many-Objective Neural Architecture Generator (POMONAG), extending DiffusionNAG through a many-objective diffusion process. POMONAG simultaneously considers accuracy, the number of parameters, multiply-accumulate operations (MACs), and inference latency. The experiments validate the performance of the proposed model.", "summary": "This paper is a direct extension based on DiffusionNAG, which can deal with multi-objective optimization in NAS. These objectives include accuracy, the number of parameters, multiply-accumulate operations (MACs), and inference latency. This motivation is good and natural, and the authors expressed their work clearly, from the motivation to the experiments results. Some details need to be clarified.", "title": null, "weaknesses": "1, The multi-objective optimization problem formulation in this work can be given first, which then can be solved by the proposed weighted factors in the reverse diffusion process. But maybe the authors can consider other ways to sovle this. For example, using four single reverse diffusion process each targeting one factor, as DiffusionNAG did, then using multi-objective optimization for further trade-off may also work well.\n\n2, The theoretical analysis should be strehghen. One objective to many objective is a breakthrough, but such process needs more analysis or discussion. Current work lacks such in-depth thinking. \n\n3, Several predictors are needed in this work, but the detailed information these predictors are missing." } ], "source_url": "https://api2.openreview.net/notes/search?term=diffusion&type=terms&content=all&group=ICLR.cc%2F2025%2FConference&source=reply&limit=2&offset=0", "total": 10000, "total_capped": true, "venue": "ICLR.cc/2025/Conference" } ``` ### Search submissions - Capability: `papers/search_submissions` - Description: Full-text search over public OpenReview submissions (papers), optionally scoped to one venue group, through GET /notes/search?source=forum. Returns one page of submissions with title, abstract, authors, author profile ids, venue label and venue id (which distinguishes accepted, withdrawn, rejected and desk-rejected submissions), keywords, TL;DR, PDF and HTML links, license, BibTeX and timestamps, plus the total hit count (the v2 index caps it at 10000) and the next offset. `api: v1` queries the legacy api.openreview.net index (venues before 2023 and dblp records; 5 requests per minute per IP) instead of the default v2 index. - Instructions: Find papers by topic, title, author or keyword, or find the forum id and venue id behind a paper. Matching is full-text term matching, not exact phrase matching; a wildcard `*` matches nothing, so a venue's complete submission list cannot be browsed here. One paper by id and its review thread are behind the Turnstile gate and are not offered. - Cost: 5 credits per call - Capability file: [Search submissions](https://firecrawl.dev/alexandria/agents/providers/openreview-net/papers/search_submissions) Accepted options: - `api` (string): Which OpenReview index to search: `v2` (api2.openreview.net, venues since about 2023; default) or `v1` (api.openreview.net, legacy venues and dblp records; rate-limited to 5 requests per minute per IP). Example: `v2` - `field` (string): Which content fields the terms are matched against (default `all`). Example: `all` - `limit` (number): Results per page, 1-100 (default 25). Example: `10` - `offset` (number): Zero-based offset of the first result; pass the previous page's `next_offset` to continue. Example: `10` - `query` (string, required): Search terms (full-text, e.g. `diffusion models`, `Yoshua Bengio` with field `authors`). Must contain a non-whitespace character. Example: `diffusion models` - `venue` (string): Venue group id to scope the search to, e.g. `ICLR.cc/2025/Conference` or `TMLR` (the `id` from list_venues / get_venue). Omit or pass `all` for every venue. Example: `ICLR.cc/2025/Conference` Response schema example: ```json { "api": "v2", "count": 3, "field": "all", "limit": 3, "next_offset": 3, "observed_at_ms": 1790993614073, "offset": 0, "query": "diffusion", "source_url": "https://api2.openreview.net/notes/search?term=diffusion&type=terms&content=all&group=ICLR.cc%2F2025%2FConference&source=forum&limit=3&offset=0", "submissions": [ { "abstract": "Hierarchical reinforcement learning (HRL) aims to solve complex tasks by making decisions across multiple levels of temporal abstraction. However, off-policy training of hierarchical policies faces non-stationarity issues because the low-level policy is constantly changing, which makes it difficult for the high-level policy that generates subgoals to adapt. In this paper, we propose a conditional diffusion model-based approach for subgoal generation to mitigate these non-stationarity challenges. Specifically, we employ a Gaussian Process (GP) prior on subgoal generation as a surrogate distribution to regularize the diffusion policy and inform the diffusion process about uncertain areas in the action space. We introduce adaptive inducing states to facilitate sparse GP-based subgoal generation, enhancing sample efficiency and promoting better exploration in critical regions of the state space. Building on this framework, we develop an exploration strategy that identifies promising subgoals based on the learned predictive distribution of the diffusional subgoals. Experimental results demonstrate significant improvements in both sample efficiency and performance on challenging continuous control benchmarks compared to prior HRL methods.", "api_version": 2, "author_ids": [ "~Vivienne_Huiling_Wang1", "~Tinghuai_Wang1", "~Joni_Pajarinen2" ], "authors": [ "Vivienne Huiling Wang", "Tinghuai Wang", "Joni Pajarinen" ], "bibtex": "@misc{\nwang2024uncertaintyregularized,\ntitle={Uncertainty-Regularized Diffusional Subgoals for Hierarchical Reinforcement Learning},\nauthor={Vivienne Huiling Wang and Tinghuai Wang and Joni Pajarinen},\nyear={2024},\nurl={https://openreview.net/forum?id=JNsac6zbg2}\n}", "created_at_ms": 1727463712418, "domain": "ICLR.cc/2025/Conference", "forum": "JNsac6zbg2", "forum_url": "https://openreview.net/forum?id=JNsac6zbg2", "html_url": null, "id": "JNsac6zbg2", "invitations": [ "ICLR.cc/2025/Conference/-/Submission", "ICLR.cc/2025/Conference/-/Post_Submission", "ICLR.cc/2025/Conference/Submission11671/-/Full_Submission", "ICLR.cc/2025/Conference/-/Withdrawn_Submission" ], "keywords": [ "Hierarchical Reinforcement Learning" ], "license": "CC BY 4.0", "modified_at_ms": 1733232781021, "number": 11671, "pdf_url": "https://openreview.net/pdf?id=JNsac6zbg2", "published_at_ms": null, "title": "Uncertainty-Regularized Diffusional Subgoals for Hierarchical Reinforcement Learning", "tldr": "We propose a subgoal generation method for HRL using conditional diffusion models and a Gaussian Process (GP) prior to address non-stationarity in off-policy training, improving subgoal regularization and exploration in uncertain areas.", "venue": "ICLR 2025 Conference Withdrawn Submission", "venue_id": "ICLR.cc/2025/Conference/Withdrawn_Submission" }, { "abstract": "Large Language Models can generate harmful content when prompted with carefully crafted inputs, a vulnerability known as LLM jailbreaking. As LLMs become more powerful, studying jailbreaking becomes a critical aspect of enhancing security and human value alignment. Currently, jailbreak is usually implemented by adding suffixes or using prompt templates, which suffers from low attack diversity. Inspired by diffusion models, this paper introduces the DiffusionAttacker, an end-to-end generative method for jailbreak rewriting. Our approach employs a seq2seq text diffusion model as a generator, conditioning on the original prompt and guiding the denoising process with a novel attack loss. This method preserves the semantic content of the original prompt while producing harmful content. Additionally, we leverage the Gumbel-Softmax technique to make the sampling process from the output distribution of the diffusion model differentiable, thereby eliminating the need for an iterative token search. Through extensive experiments on the Advbench and Harmbench, we show that DiffusionAttacker outperforms previous methods in various evaluation indicators including attack success rate (ASR), fluency, and diversity.", "api_version": 2, "author_ids": [ "~Hao_Wang82", "~Hao_Li62", "~Junda_Zhu1", "~Xinyuan_Wang2", "~Chengwei_Pan1", "~Minlie_Huang1", "~Lei_Sha1" ], "authors": [ "Hao Wang", "Hao Li", "Junda Zhu", "Xinyuan Wang", "Chengwei Pan", "Minlie Huang", "Lei Sha" ], "bibtex": "@misc{\nwang2024diffusion,\ntitle={Diffusion Attacker: Diffusion-Driven Prompt Manipulation for {LLM} Jailbreak},\nauthor={Hao Wang and Hao Li and Junda Zhu and Xinyuan Wang and Chengwei Pan and Minlie Huang and Lei Sha},\nyear={2024},\nurl={https://openreview.net/forum?id=u08UxVNdIo}\n}", "created_at_ms": 1727194565215, "domain": "ICLR.cc/2025/Conference", "forum": "u08UxVNdIo", "forum_url": "https://openreview.net/forum?id=u08UxVNdIo", "html_url": null, "id": "u08UxVNdIo", "invitations": [ "ICLR.cc/2025/Conference/-/Submission", "ICLR.cc/2025/Conference/-/Post_Submission", "ICLR.cc/2025/Conference/Submission3818/-/Full_Submission", "ICLR.cc/2025/Conference/-/Withdrawn_Submission" ], "keywords": [ "LLM safety; LLM jailbreak; Diffusion Language Model; Gumbel Softmax;" ], "license": "CC BY 4.0", "modified_at_ms": 1734334540398, "number": 3818, "pdf_url": "https://openreview.net/pdf?id=u08UxVNdIo", "published_at_ms": null, "title": "Diffusion Attacker: Diffusion-Driven Prompt Manipulation for LLM Jailbreak", "tldr": null, "venue": "ICLR 2025 Conference Withdrawn Submission", "venue_id": "ICLR.cc/2025/Conference/Withdrawn_Submission" }, { "abstract": "Despite advances in diffusion-based text-to-music (TTM) methods, efficient, high-quality generation remains a challenge. We introduce Presto!, an approach to inference acceleration for score-based diffusion transformers via reducing both sampling steps and cost per step. To reduce steps, we develop a new score-based distribution matching distillation (DMD) method for the EDM-family of diffusion models, the first GAN-based distillation method for TTM. To reduce the cost per step, we develop a simple, but powerful improvement to a recent layer distillation method that improves learning via better preserving hidden state variance. Finally, we combine our step and layer distillation methods together for a dual-faceted approach. We evaluate our step and layer distillation methods independently and show each yield best-in-class performance. Our combined distillation method can generate high-quality outputs with improved diversity, accelerating our base model by 10-18x (230/435ms latency for 32 second mono/stereo 44.1kHz, 15x faster than the comparable SOTA model) — the fastest TTM to our knowledge.", "api_version": 2, "author_ids": [ "~Zachary_Novack1", "~Ge_Zhu1", "~Jonah_Casebeer1", "~Julian_McAuley1", "~Taylor_Berg-Kirkpatrick1", "~Nicholas_J._Bryan1" ], "authors": [ "Zachary Novack", "Ge Zhu", "Jonah Casebeer", "Julian McAuley", "Taylor Berg-Kirkpatrick", "Nicholas J. Bryan" ], "bibtex": "@inproceedings{\nnovack2025presto,\ntitle={Presto! Distilling Steps and Layers for Accelerating Music Generation},\nauthor={Zachary Novack and Ge Zhu and Jonah Casebeer and Julian McAuley and Taylor Berg-Kirkpatrick and Nicholas J. Bryan},\nbooktitle={The Thirteenth International Conference on Learning Representations},\nyear={2025},\nurl={https://openreview.net/forum?id=Gj5JTAwdoy}\n}", "created_at_ms": 1727290922703, "domain": "ICLR.cc/2025/Conference", "forum": "Gj5JTAwdoy", "forum_url": "https://openreview.net/forum?id=Gj5JTAwdoy", "html_url": null, "id": "Gj5JTAwdoy", "invitations": [ "ICLR.cc/2025/Conference/-/Submission", "ICLR.cc/2025/Conference/-/Post_Submission", "ICLR.cc/2025/Conference/Submission4966/-/Full_Submission", "ICLR.cc/2025/Conference/Submission4966/-/Rebuttal_Revision", "ICLR.cc/2025/Conference/-/Edit", "ICLR.cc/2025/Conference/Submission4966/-/Camera_Ready_Revision" ], "keywords": [ "music generation", "diffusion distillation", "diffusion", "diffusion acceleration", "text-to-music generation", "layer dropping" ], "license": "CC BY 4.0", "modified_at_ms": 1739380024687, "number": 4966, "pdf_url": "https://openreview.net/pdf?id=Gj5JTAwdoy", "published_at_ms": 1737562384096, "title": "Presto! Distilling Steps and Layers for Accelerating Music Generation", "tldr": null, "venue": "ICLR 2025 Spotlight", "venue_id": "ICLR.cc/2025/Conference" } ], "total": 1335, "total_capped": false, "venue": "ICLR.cc/2025/Conference" } ```