New medical LLM, PathChat 2, can talk to pathologists about tumors, offer diagnoses
What unique capabilities does PathChat offer to pathologists?
PathChat is a pathology-specific large language model that can accurately identify, assess, and diagnose tumors and other serious conditions. It performs significantly better than leading models on multiple-choice diagnostic questions and can generate clinically relevant responses to open-ended inquiries3. PathChat can serve as a consultant for human pathologists, offering informed pathology advice and supporting research.
How does PathChat perform compared to other LLMs like GPT-4 and LLaVA?
PathChat outperforms GPT-4 and LLaVA in both image-only and clinical context prompts. In image-only evaluation, PathChat scores more than 52% better than LLaVA and more than 63% better than LLaVA-Med. When provided with clinical context, PathChat performs 39% better than LLaVA and nearly 61% better than LLaVA-Med. It also performs more than 53% better than GPT-4 with image-only prompts and 27% better with prompts providing clinical context.
What training methods were used to develop PathChat?
PathChat was developed by adapting a vision encoder for pathology, combining it with a pre-trained large language model, and fine-tuning with visual language instructions and question-answer turns4. The model was trained on a vast dataset to understand human pathology, and it can interactively assist both researchers and pathologists across various areas of pathology, tasks, and scenarios.