You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Does the work provide new insights, deepen understanding, or highlight important properties of existing methods?
53
56
Is it clear how this work differs from previous contributions, with relevant citations provided?
54
57
Does the work introduce novel tasks or methods that advance the field?
55
58
Does this work offer a novel combination of existing techniques, and is the reasoning behind this combination well-articulated?
56
59
As the questions above indicates, originality does not necessarily require introducing an entirely new method.
57
60
Rather, a work that provides novel insights by evaluating existing methods, or demonstrates improved efficiency, fairness, etc. is also equally valuable.
58
61
62
+
59
63
# Scores
60
-
Try to be specific and detailed in your assessment. Try not to set the same score for all the dimensions.
61
-
62
-
Quality: Based on what you discussed in the “Quality” section, please assign the paper a numerical rating on the following scale to indicate the quality of the work.
63
-
4 = excellent
64
-
3 = good
65
-
2 = fair
66
-
1 = poor
67
-
68
-
Clarity: Based on what you discussed in the “Clarity” section, please assign the paper a numerical rating on the following scale to indicate the clarity of the paper.
69
-
4 = excellent
70
-
3 = good
71
-
2 = fair
72
-
1 = poor
73
-
74
-
Significance: Based on what you discussed in the “Significance” section, please assign the paper a numerical rating on the following scale to indicate the significance of the paper.
75
-
4 = excellent
76
-
3 = good
77
-
2 = fair
78
-
1 = poor
79
-
80
-
Originality: Based on what you discussed in the “Originality” section, please assign the paper a numerical rating on the following scale to indicate the originality of the paper.
81
-
4 = excellent
82
-
3 = good
83
-
2 = fair
84
-
1 = poor
64
+
Try to be specific and detailed in your assessment.
65
+
Try not to set the same score for all the dimensions.
66
+
The scores for all dimensions should be independent of each other.
67
+
Scores should rely on strengths and weaknesses of the paper in each dimension.
68
+
If there are many substantial weaknesses, the score should be low.
69
+
85
70
86
71
# Questions
87
-
Please list up and carefully describe questions and suggestions for the authors, which should focus on key points (ideally around 3–5) that are actionable with clear guidance.
88
-
Think of the things where a response from the author can change your opinion, clarify a confusion or address a limitation.
89
-
You are strongly encouraged to state the clear criteria under which your evaluation score could increase or decrease.
90
-
This can be very important for a productive rebuttal and discussion phase with the authors.
72
+
List 3-5 key actionable questions/suggestions.
73
+
Focus on points where author response could change your opinionor clarify confusion.
74
+
State clear criteria for your score changes.
75
+
91
76
92
77
# Limitations
93
78
Have the authors adequately addressed the limitations and potential negative societal impact of their work?
94
-
If so, simply leave “yes”; if not, please include constructive suggestions for improvement.
79
+
Please include constructive suggestions for improvement.
95
80
In general, authors should be rewarded rather than punished for being up front about the limitations of their work and any potential negative societal impact.
96
81
You are encouraged to think through whether any critical points are missing and provide these as feedback for the authors.
97
82
98
83
99
-
# Overall
100
-
Please provide an "overall score" for this submission. Choices:
101
-
6: Strong Accept: Technically flawless paper with groundbreaking impact on one or more areas of AI, with exceptionally strong evaluation, reproducibility, and resources, and no unaddressed ethical considerations.
102
-
5: Accept: Technically solid paper, with high impact on at least one sub-area of AI or moderate-to-high impact on more than one area of AI, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
103
-
4: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
104
-
3: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
105
-
2: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
106
-
1: Strong Reject: For instance, a paper with well-known results or unaddressed ethical considerations
1: Strong Reject - Known results or unaddressed ethical issues
91
+
92
+
Don't be afraid to use 1, 2, 5, and 6.
93
+
107
94
108
95
# Confidence
109
-
Please provide a "confidence score" for your assessment of this submission to indicate how confident you are in your evaluation. Choices
110
-
5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
111
-
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
112
-
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
113
-
2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
114
-
1: Your assessment is an educated guess. The submission is not in your area or the submission was difficult to understand. Math/other details were not carefully checked.
96
+
5: Absolutely certain, very familiar with related work, checked math/details carefully
97
+
4: Confident but not certain, unlikely missed something
98
+
3: Fairly confident, possibly missed parts or unfamiliar with some work, details not carefully checked
99
+
2: Willing to defend but likely missed central parts, details not checked
100
+
1: Educated guess, not your area or hard to understand, details not checked
101
+
115
102
116
103
# Format issues
117
104
Find problems with the paper formatting. Report them separately.
118
105
119
106
# Result
120
107
Return the result as a JSON object in the following format:
121
108
{
122
-
"summary": "Summary of the paper",
123
-
"quality": {
124
-
"strengths": ["...", "..."],
125
-
"weaknesses": ["...", "..."],
126
-
"score": ...,
127
-
},
128
-
"clarity": {
129
-
"strengths": ["...", "..."],
130
-
"weaknesses": ["...", "..."],
131
-
"score": ...,
132
-
},
133
-
"significance": {
134
-
"strengths": ["...", "..."],
135
-
"weaknesses": ["...", "..."],
136
-
"score": ...,
137
-
},
138
-
"originality": {
139
-
"strengths": ["...", "..."],
140
-
"weaknesses": ["...", "..."],
141
-
"score": ...,
142
-
},
143
-
"questions": ["Questions and suggestions for the authors", "..."],
144
-
"limitations": ["Limitations of the paper", "..."],
145
-
"overall": {
146
-
"strengths": ["...", "..."],
147
-
"weaknesses": ["...", "..."],
148
-
"score": ...,
149
-
},
150
-
"confidence": {
151
-
"description": "Confidence score and its description",
0 commit comments