All that glitters is not gold: Importance of rigorous evaluation of proteochemometric models
Polina Avdiunina, Shamieraah Jamal, Filipp Gusev, Olexandr Isayev
Highlight
Proteochemometric models (PCM) are used in computational drug discovery to leverage both protein and ligand representations for bioactivity prediction.
Abstract
Proteochemometric models (PCM) are used in computational drug discovery to leverage both protein and ligand representations for bioactivity prediction. While machine learning (ML) and deep learning (DL) have come to dominate PCMs, often serving as scoring functions, rigorous evaluation standards have not always been consistently applied. In this study, using kinase-ligand bioactivity prediction as a model system, we highlight the critical roles of dataset curation, permutation testing, class imbalances, data splitting strategies, and embedding quality in determining model performance. Our findings indicate that data splitting and class imbalances are the most critical factors affecting PCM performance, emphasizing the challenges in generalizing ability of ML/DL-PCMs. We evaluated various protein-ligand descriptors and embeddings, including those augmented with multiple sequence alignment (MSA) information. However, permutation testing consistently demonstrated that protein embeddings contributed minimally to PCM efficacy. This study advocates for the adoption of stringent evaluation standards to enhance the generalizability of models to out-of-distribution data and improve benchmarking practices.
Keywords
Cite This Paper
@article{Avdiunina2025a,
author = {Avdiunina, Polina and Jamal, Shamieraah and Gusev, Filipp and Isayev, Olexandr},
title = {All that glitters is not gold: Importance of rigorous evaluation of proteochemometric models},
year = {2025},
doi = {10.26434/chemrxiv-2025-vbmgc},
url = {http://dx.doi.org/10.26434/chemrxiv-2025-vbmgc},
publisher = {American Chemical Society (ACS)},
keywords = {machine learning, deep learning, drug discovery},
researchAreas = {drug-discovery},
highlight = {Proteochemometric models (PCM) are used in computational drug discovery to leverage both protein and ligand representations for bioactivity prediction.}
} Copied to clipboard!
Related Research Areas
Related Publications
Towards chemical accuracy for alchemical free energy calculations with hybrid physics-based machine learning / molecular mechanics potentials
(2020)
Towards chemical accuracy for alchemical free energy calculations with hybrid physics-based machine learning / molecular mechanics potentials.
Integrating QSAR modelling and deep learning in drug discovery: the emergence of deep QSAR
Nat. Rev. Drug Discov., 23, 141–155 (2024)
Generative and reinforcement learning approaches for the automated de novo design of bioactive compounds
Communications Chemistry, 5 (2022)
AbstractDeep generative neural networks have been used increasingly in computational chemistry for de novo design of molecules with desired properties.
Crowdsourced mapping of unexplored target space of kinase inhibitors
Nature Communications, 12 (2021)
Abstract Despite decades of intensive search for compounds that modulate the activity of particular protein targets, a large proportion of the human kinome remains as yet undrugged.
Teaching a neural network to attach and detach electrons from molecules
Nature Communications, 12 (2021)
Abstract Interatomic potentials derived with Machine Learning algorithms such as Deep-Neural Networks (DNNs), achieve the accuracy of high-fidelity quantum mechanical (QM) methods in areas traditionally dominated by empirical force fields and allow performing massive simulations.