To Publish, or Not to Publish
Should medical companies publish and share their research in scientific publications and conferences on a regular basis.
Originally published on Medium
TL;DR
YES. You should publish.
Photo by MILKOVÍ on Unsplash
In this post, I am going to discuss whether ML groups of medical startups should routinely work on publishing papers. I will try to show the benefits and disadvantages in order to help determine if publications should be prioritized over other projects.
First, a disclaimer. Everything is written from my perspective, which is based on my personal experience, working in ML research teams in healthcare startups. I think it applies to a variety of domains, but well — who knows.
A fascinating perspective called “Stealth research: Lack of peer‐reviewed evidence from healthcare unicorns” was recently published in the European Journal of Clinical Investigation. The authors discuss the issue of Stealth Research and state that:
“Healthcare start‐ups are a major force in biomedical innovation, positioned to make revolutionary discoveries.”
but show concern that:
“Most of the highest‐valued start‐ups in healthcare have a limited or non‐existent participation and impact in the publicly available scientific literature.”
It’s a very interesting read, and I couldn’t agree more. In today’s research environment, when state-of-the-art results are frequently achieved, and code is shared in open source repositories, medical companies can’t stay behind. They must consider and find a way to share their knowledge and experience within the scientific community. I know; there is always the fear of revealing your secrets, especially in the early days of the company. Still, if you find that much of the existence of your company is hanging on one or two “secret sauce” algorithms, that are not to be shared to some extent, your company is in trouble. There are lots of smart people out there working to promote this field forward, and someone can or will figure it out. You can only profit in the long run from publishing, and you should make it a habit and routine under several considerations, which I will describe here.
Very broadly let’s divide publications possibilities into four options:
- Top tier journals [Nature, Science, The Lancet, …]
- Clinical journals or clinical conferences [Journal Rankings]
- Technical oriented medical conferences [MICCAI, MIDL, MLHC, …]
- Top technical conferences [CVPR, ECCV, ICLR, NeurIPS, …]
All of these options benefit the company, team, and also the culture, but in a different degree of impact.
Let’s go through each one of these possibilities and try to understand what are the pros and cons of working toward publishing in these venues.
Top tier journals
The holy grail publication for medical companies. The one most of the companies and researchers want to achieve. A paper you can proudly bring home to make your parents proud.
Such publication validates your research and helps your go-to-market [AI] strategy. It shows that your work has been reviewed and verified outside the inner company circles and was approved. This fact will be helpful when approaching new customers and investors.
It is also beneficial in terms of handling competing companies and improving the company moat — raising the entrance bar to the field.
Publishing in top tier journals also puts the company under the spotlight of reporters and attracts excellent public relations. The PR machine will do the work for you, describing the main key points of the paper in major magazines, social networks, interviewing the authors, etc. Eventually, this creates hype around the company and leads traction to the product by a new segment of users.
Now comes the bad parts.
Such publication requires a lot of effort and time. It often comes after an already long journey of a big project that got to an end with some breaking or innovating results, and now instead of releasing some steam, relax and move to the next project, this will hold many team members back from progressing to other tasks. Working on a paper will hold up one of the main company verticals from moving forward. This fact is not suitable for all companies and highly depends on the stage the company is at. You should think it through before starting this adventure.
It also assumes your team has the skillset and can handle the relevant tasks: writing a scientific paper, reproducing results, designing figures, reviewing the materials, etc.
Also, you should consider how this might affect the team morale, as a group and on an individual level. First, the concern of rejection, which spoils the morale and shakes the ship confidence after the significant effort already invested. Second, the reviewing process can be very long, and it can take weeks until getting the referees review back. At the same time, team members are already occupied with other tasks, and now need to contact switch back to old experiments in order to add the value of a missing statistic. Of course, this will be done in some ancient commit, which now does not work [BTW, you should use docker containers and wrap your system close to submission to be prepared for this scenario].
The last thing you should give your attention to, which might become an issue, is the authors’ order of appearance in the paper. It can turn into a pain point in case egos are involved — it is hard to make everyone happy, and often someone will be eventually unsatisfied. Think about this in advance.
Clinical journals or clinical conferences
In this type of publication, I mean either known journals of a specific topic in healthcare or highly valued conferences that publish a proceeding or something similar.
I think the best way is to run this with strong collaborators that lead the effort and propose the research question. The team, in this case, support and help to push the collaboration forward. This way the heavy lifting is divided between an outside group and your team. Both parties enjoy sharing and seeing another perspective of the domain they are working in. Also, a publication with known researchers and opinion leaders in your field will contribute to the company’s credibility and involvement in the industry and academia.
You should pay attention not to take too many collaborations at the same time, as they tend to overlap with the current workflow and slow down the team. Collaborators often work at a different pace and different time zone, and if not appropriately managed can lead to team frustration. It is hard to point a magic number of collaborations to work on at the same time, and also depends on the team size and who handles these projects in the long run. Additionally, it is hard to reject collaborations from an important expert, even if you already busy with others, but try to limit yourself to the best ideas, which will result in meaningful publications.
Some of these collaborations will require work from team members but put them in the non-front seat. This is inevitable for some collaborations, as researchers might want to handle the research and use your “services” when they are needed. However, having team members working on this and only sending reports can be very frustrating. I suggest that team members will be involved in the research all the way. And have them be the contact person talking to the collaborators, which will lead to more personal relationships. Besides, credit and recognition are important. If you have teammates working for others “in the dark” you will end up with low morale and frustration.
Technical oriented medical conferences
This type of publication is a great middle-ground. It often requires less work than other alternatives and fits many projects the team has been working on already.
Submitting papers to these conferences brings a lot of happiness to the team, and once accepted will increase team morale. It will spark other team members to push their projects to high quality in order to be relevant for this type of publication.
It can help to expose the company to the community and also recruiting new talents to the team and attract researchers with interest in publications.
On a more fuzzy viewpoint, it creates a feeling of excellence, which comes from the understanding that the work the team is doing does not only relates to the company domain but is relevant for others in adjacent domains.
I think this type of publication benefits both the company and the team. As the return of investment is high, and it requires less work than other options, I think teams should engage in such publication at least once a year.
Top technical conferences
If top tier journals were companies holy grail, these conferences are in the wish list of every ML researcher. We all read and work with these publications on a train-to-train basis.
Not by chance, those who submit and publish in these conferences are researchers from academia, top talent in the field or teams from big companies. It might require exploring problems that are not in direct connection to the scope of the company and often requires working with public datasets. Startups companies usually don’t have the privilege and capacity to allocate resources for such projects.
One of the advantages these conferences have, which is also right for technical oriented medical conferences, is the pre-known deadlines. Deadlines are good, especially for research teams. The deadlines are announced on the conference website, and you can adjust the team projects and set a fixed time for such publication work.
Another advantage is the short process review in these conferences compare to journals. It is basically a rejection or acceptance response [with a short period to rebuttal and update the paper].
Publications in these conferences position the team in the “big league” and brand the group as a prestigious one. However, this type of publication requires lots of computing power, which costs a lot of money, especially if you are using cloud computing. It could pile up to ten of thousands of dollars and “eat” the team computing budget. Additionally, it requires significant time and focused work on the small details that are sometimes ignored during research, like concentrating on improving 1–2% accuracy of a publicly available known dataset.
As a final note, I would like to emphasize three main points.
(1) Aim high, prepare it well and start by submitting the work to a higher level target then what you think you are worth.
(2) I think you always learn something new while working on a paper by building a more intimate understanding of your datasets and models. There is an amazing feeling when writing an article, that pushes further the quality of your work and makes it better for yourself and the company.
(3) There are many tips and different moves you can do if publications are a goal for your team. I might write another post about it, but the most important tip is getting the right experts at your side. You will need a supportive group of advisors to help push this ship to the right port. Don’t underestimate this. A word in the right direction from someone who published numerous times in the past is super important. I had the honor and privilege to work with a few super talented and smart advisors and keep learning from them today on every discussion.
So where are you planning to publish next?