The 'FaceSurvival' AI tool suggests the area under the eyes and around the bridge of the nose are the most vital parts of the face to assess when estimating how long they might have left to live.
These metrics applied across many different types of cancers, varying in severity, including ones that had spread to the patient's bones or brain.
Most of the cancer patients, who were from the US and the Netherlands, were in their 50s or 60s, but some were children, while others were in their 90s.
Doctors already examine patients and run comprehensive checks and tests to judge how unwell they are, and what their prognosis might be.
The AI tool, which has a patent application pending, is designed to help doctors doing this work, not replace them.
Lead researcher Dr Raymond Mak, from Mass General Brigham in Boston, explained: "As the population ages, physicians need better ways to estimate physical reserve and frailty before recommending intensive treatments such as radiation, surgery or systemic therapy.
"Our goal is to make face photograph-based health assessment more quantitative, consistent and scalable so it can support personalised treatment decisions rather than replace physician judgment."
Mak and colleagues are also evaluating an AI tool for predicting biological age, or how old someone looks, as opposed to their birthday age.
The 'FaceAge' tool suggests the skin below the nose and near the mouth - the so-called nasolabial folds and marionette lines - plus the forehead (frown lines), temples (crows' feet) and cheeks are a good indicator of how fast we age.
Some lifestyle habits can speed that process up, like heavy alcohol use, drug use and smoking.
Mak says the tool was "surprisingly consistent" across common real-world photos despite variations in pose, lighting, facial expression and make-up.
"It was also stable across most tested plastic surgery procedures, with only a small shift observed after rhinoplasty [a nose job]," he says.
But the researchers stress they need much more data and photos of faces, particularly from lots of different ethnic groups, to determine how well the tools work. They are still very much under development.
The research team is inviting the public to take part, external and receive a personalised biological age estimate by uploading their own facial photo the research team.
Prof Miroslaw Bober, an expert in video processing and machine learning at the University of Surrey in the UK, says any claims that the tool could track ageing or lifestyle changes over time would be "premature".
"This is a promising proof of concept, but it is not ready for clinical use," he said.
"The tool still needs testing against doctors' own judgement and across different hospitals and patient groups."
And he warned: "Predicting an outcome is not the same as explaining it."