Final Project Proposal
COMS 6998: Analysis of Networks & Crowds
Final paper: what to do
The rules are finalized below to help guiding you in the final weeks. But there are no changes: the spirit is always the same, which is to give you a chance to draw a new path in this domain for yourself. What may the most distinguishing feature of our evaluation is that we are judging you on a research proposal rather than a research result. It seems much more interesting as time is short, it’s more interesting to use the little time to reflect on a good topic than rush to implement an incremental step.
What are we evaluating in this final paper?
Your goal: Formulating a clear research problem and a plausible plan to explore it.
You have made hard proofs, you have tried your best to understand real data and polish algorithms. Now you will demonstrate that you can create a good proposal for more research in that domain. This is often the first real step of a research career (I understand many expect research to start by learning the ropes on a given topic chosen for them. While this is not bad, until you get a chance to propose, you’re not really searching! There is no reason to delay). In my experience, even if that’s not said explicitly, every advisor expect students from day1 in proposition mode. By that one does not mean stubbornly motivated by a single idea obviously, but able to create and develop new paths from a given status quo. That’s what we want you to do (and as opposed to a real fellowship/summer project etc., you won’t necessary have to do the work 🙂 ).
NB: Some of you may have already results, and that’s great. This will be counted as extra-credit! But some may just have been waiting for the push, so here you go.
How we will evaluate it?
We will judge the idea proposed according to three dimensions -absolute merit (score 1-5): is the problem clear? is the research impactful for social network and big data application? -relative merit (score 1-3): is it well connected to current research in a way that is articulated and offer a significant novelty. Some candidness is acceptable. – soundess of approach (1-3): how many and how uncertain are the connections needed for that idea to lead to an advance. The final score is multiplicative, to account for the fact that a good research proposal cannot be prioritizing one of those dimensions at the expense of even a single of the others. That multiplicative rule is meant to reproduce reality of actual success in research.
How is your grade computed? Designing a good scoring method for your final paper amounts to finding an accurate estimate on the change of a research proposal. It’s not easy, and it’s not
程序代写 CS代考 加微信: cstutorcs
necessarily “fair” in the usual sense (i.e., I spend x minutes out of y available and I received x/y). During some critical phases, an additional effort that may seem small actually creates a multiplying factor to build your research. That is the effect I have captured in the grading scheme below.
Your grade is a multiplication of 3 scores:
• Novelty/Challenge (graded on 6): Assuming the project is sound and approach proposed successful, how far have we progressed on an important problem that is either new or known to be difficult? – 0: plagiarism / generative-AI from obvious prompts – 1: no difference between proposed projects and bread-butter – 2: difference is doubtful, unclear it is an open question – 3: variation/combination of known concepts – 4: sidestep from a known approach that poses new questions – 5: creative/novel approach, extends a challenging case – 6: blockbuster novelty, could break many barriers
• Maturity/Significance (graded on 4): Taking the contribution for what it is and assuming it works, does the significance of the progress achieved stand out? – 0: proposed approach is contradicted by well known facts – 1: presents no knowledge of related work outside surface – 2: related work is loosely covered (partially, high level) – 3: some significance w.r.t. the
state of the art appear – 4: significance of the contribution is thoroughly developed
• Clarity/Soundness (graded on 10): What is the chance that the project lead to a scientific
discovery? – 0: expecting success fit Einstein’s definition of insanity – 1: project mostly motivational, success-path unidentified – 2-3: the outputs of the projects are likely uninterpretable – 4-5: a potential success, but unlikely via major limitation – 6: a discovery would be surprising but can’t be excluded – 7: a good bet: at least some moderate progress to expect – 8: a great bet: major progress not overall improbable – 9-10: a sure bet, sound path to positive/negative discovery
So the maximum theoretical score is 240pt. Reaching for that maximum means contradicting the law of intellectual physics (i.e., formulating a sure bet on a major novel approach, while having shown how it will revolutionize multiple fields). That is not a bug! It’s important that you are aware of the extreme significance of success that seems partial but are truly decisive. You need to see in practice that a 100% grade in a research project makes no sense. In practice, any score above 100pt is impressive (or lucky :)), and above 150pt is truly outstanding.
• A good incremental step would combine moderate novelty (more like variation of known concept (3), with good coverage of the literature and why this new angle is significant (3) but perhaps would miss a few things, and it will have a solid plan (8). That’s already a 96pt.
• A “boring’ ’ project that would combine known concept (3), do only a partial literature search (2) and present an experiment that is not likely to bring much advance (6) receives 36pt.
Note in the examples below that there is almost a 1-3 ratio and that from the “boring” starting point, any increment is extremely important. Making a boring idea more mature, show sign of possible progress and addresses concerns passed this point to rise to another level is the path to success.
• An “inspiring’ ’ project that proposes a creative approach (5) that is well motivated by related work (4), even if it is unlikely to work (5) still receives 100pt. That’s because research reward risk (provided the approach is not obviously flawed or hopeless).
CS Help, Email: tutorcs@163.com
• A real new idea (6) even with poor maturity (2) on which a discovery can’t be completely excluded (5) already receives 72pt.
Again any progress from there (especially on maturity, which is attainable) would make a spectacular score.
What are the scores in last year’s project? Average of submitted papers is 60pt which is also the median
• 6 projects got more than 100pt (including 2 above 150pt). Congrats to you!
• Nobody scored 0 in any dimensions, and nobody scored score 9-10 in soundness. However,
except for those cases, all dimensions mattered and for any of those there has been a project with this score! It seems that overall the distinguishing feature of top project is combining a big challenge/novelty with impeccable state of the art.
• the score distribution (see attached picture) is almost uniform on [0;150pt], which is somehow an interesting findings.
/FinalPaperGrade.png
If you are wondering how far you are in those dimensions with your project ideas, I recommend reading and answering the Heilmeier’s 9Q (see at the end of this docuement). It’s a commonly used canvas for judging research proposal in multiple institutions like NSF, DARPA, ERC, Google, MSR … If you have a satisfactory answers to all, you’re on the good track (or too optimistic 🙂 it happens), otherwise, it might help you choose how to exert effort to address what’s missing.
Where can I find some ideas for projects?
For starters, as shared during the semester, we recommend high visibility events: TheWebConf (https://www2024.thewebconf.org/), The ACM EC Conference (https://ec24.sigecom.org/index.html), ACM EAAMO Conference (https://conference.eaamo.org/), as well as topical workshops, such as the Simons institute (https://simons.berkeley.edu/homepage)
Those are great events that typically include the “core” of social network theory with impacts, but some of those topics vastly expands in different venues, often in related computing conferences (KDD, ICWSM, ICML, NeurIPS) or multidisciplinary conferences (FAcct, AAAI AIES) and occasionally in many others. As long as you see a way to connect your project to some efforts related in those areas you are probably entirely in scope. Please do not assume that an original idea you have is out of scope without requesting help, and do not hesitate to ask doing the connection, as it’s often the most interesting and difficult part.
What format to use for the final paper?
We are expecting pdf, 4p in the ACM format https://www.acm.org/publications/proceedings-template
Alternative formats that are roughly equivalent are fine (but why would you do that?). We expect 4p to include the references (so 3p to 3.5p of actual text), but we won’t have any problem to include references on another page. If you have more to say, you could have a longer paper as long as it has the same density/quality. We recommend you to include anything that takes more space as an
Programming Help, Add QQ: 749389476
appendix not to disturb your problem formulation. Any sections that will be about “results” can be included in the body of the text as you want, no problem at all.
You can also make it shorter, as long as the problem is clearly formulated and its merits are well justified. But obviously it’s even harder to show maturity and novelty in a shorter form (esp. without the “no space for that” card), so do that at your own risk.
When is the paper due?
The due date for the paper is Thursday Dec. 21st. Please contact me if you need extra days (especially in case you have multiple final exams in the way).
Looking forward to helping you with those proposals,
• Augustin
9 questions to ask yourself (and answer) about your project 1-2-3:
Core Question 1:
– What are you trying to do ?
– Articulate your objective using absolutely no jargon. – What is the problem? – Why is it
Question 2:
• How is it done today and what are the limits of current practice? Question 3:
• What is new in your approach and why do you think it will be successful? 4-5-6: Impact of the contribution
Question 4:
• Who cares? Question 5:
• If you are successful, what difference will it make?
• What impact will success have? How will it be measured?
Question 6:
• What are the risks and payoffs?
7-8-9: Plans and evaluations Question 7:
• How much will it cost? Question 8:
• How long will it take? Question 9:
• What are the midterm and final “exams” to check for success? – How will progress be measured?