Weekly reading
The big news of the week, of course, was the release of GPT-4 by Open AI. It scores in the 90th+ percentile for many standardized professional exams (90th percentile on the bar exam, for example).
You can read the paper which specifically states they won’t release additional technical details due to safety and competitive concerns, including “no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar.” It does describe that the loss has dramatically decreased (remember that less loss is better, meaning it’s closer to the ideal): 
It can respond to images or text (the vision/no-vision option mentioned in the bar graph about standardized test scores), which is pretty incredible.
Large language models often fail at adversarial tasks, meaning questions that have an incorrect answer that is statistically appealing. GPT-4 increases the percentage of these tasks it gets right by about 19% depending on the domain compared to GPT-3.
The safety section is worth reading to understand what the world experts in AI are worried about. Open AI gathered 50 of them to comment on possible nefarious uses. I thought it was interesting that it uses human-designed rubrics to tell the system how to respond to prompts that are “disallowed” (ie, how to synthesize dangerous chemicals) and “sensitive” (ie, medical advice).
Fun Fact
Researchers at Hopkins recently diagrammed every neural connection in the brain of a larval fruit fly and found “circuit features that were strikingly reminiscent of prominent and powerful machine learning architectures.”
This Week’s Top Stories
- Researchers in Vancouver trained neural NLP models and were able to predict survival using the first oncologist note with accuracy similar or better than existing tools. Unlike previous models, this model was not limited to a single type of cancer. As a palliative care doctor, this is extremely helpful; previous studies have shown oncologists to overestimate their patients’ survival by 300%.
- Medicare is using AI to predict how long Medicare Advantage recipients should be using services, then issuing denials. Seems like they still have some work to do on nuance in the system.
- Cool things Google Health AI is doing:
- Creating a tool to read physician handwriting. I really could have used that when I was a pharmacy tech 20 years ago.
- Improving chest X-ray–based TB screenings in Africa
- Trialing AI-assisted ultrasound for pregnant women in Africa
- Exploring breast ultrasound in Taiwan