CS 8803 DMM: Data Management and Machine Learning (Fall 2020)

Paper Review

For each research paper, the student must write a paper review, and submit the paper review in a given deadline to the instructor and the TA via Slack.

Submission Guideline

• The review must be submitted to the Instructor (Xu Chu) and the TA (Hantian Zhang) in the course slack.
• Late submissions will not be accepted.
  
• Given unpredictable workloads students may have during the semester, you are allowed to miss at most three reviews during the semester. 

Review Structure

Each review must contain the following information:

• Summary: (An overview of the main idea and contributions in one paragraph)
• What is this paper about?
  
• What is the main contribution?
  
• Describe the main approach & results. Just facts, no opinions yet.
  
• Summarize the insights resulting from the empirical study.
• Strength: (State at least three strengths of the paper)

• Is there a new theoretical insight? Or a significant empirical advance?
• Did they solve a standing open problem? Or a good formulation for a new problem? Or a faster/better solution for an existing problem?
• Any good practical outcome (code, algorithm, etc)?
• Are the experiments well executed?
  
• Useful for the community in general?
• Other types of strengths you may think of ... 
• Weakness: (State at least three weaknesses of the paper.)
• What can be done better?
• Any missing baselines? Missing datasets?
• Any odd design choices in the algorithm not explained well?
• Quality of writing?
• Is there sufficient novelty in what they propose?
• Minor variation of previous work?
• Why should anyone care? Is the problem interesting and significant?
• Other types of strengths you may think of ...
• Two discussion points: (To be brought by you in classroom discussion)
• How does this relate to other papers we have read?
  
• What are the next research directions in this line of work?
  
• What (directly or indirectly related) new ideas did this paper give you?
  
• What would you be curious to try?
  
• What experiments can deepen our understanding?
  
• How can your own research benefit from the insights provided in the paper?
  
• Why is the paper not interesting?
  
• What is the paper’s potential impact on the field?

Grading

Each review will be evaluated on a 100 points scale:

• 20 points for summary
• 20 points for strengths
  
• 20 points for weakness
• 20 points for discussion points
• 20 points for clarity and cogent statements
• Bonus: 10 points for exceptional reviews with new insights that can inspire future work