pocketz.doi.bio/2026-01-11_181713_nature_s42256-025-00993-0

Benchmarking AI-powered docking methods from the perspective of virtual screening | Nature Machine Intelligence

URL: https://www.nature.com/articles/s42256-025-00993-0

Saved: 2026-01-11T02:17:18.564Z

Analysis Published: 13 February 2025 Benchmarking AI-powered docking methods from the perspective of virtual screening Shukai Gu, Chao Shen, Xujun Zhang, Huiyong Sun, Heng Cai, Hao Luo, Huifeng Zhao, Bo Liu, Hongyan Du, Yihao Zhao, Chenggong Fu, Silong Zhai, Yafeng Deng, Huanxiang Liu, Tingjun Hou & Yu Kang 

Nature Machine Intelligence volume  7,  pages 509–520 (2025)Cite this article

6153 Accesses

18 Citations

6 Altmetric

Metrics details

Abstract

Recently, many artificial intelligence (AI)-powered protein–ligand docking and scoring methods have been developed, demonstrating impressive speed and accuracy. However, these methods often neglected the physical plausibility of the docked complexes and their efficacy in virtual screening (VS) projects. Therefore, we conducted a comprehensive benchmark analysis of four AI-powered and four physics-based docking tools and two AI-enhanced rescoring methods. We initially constructed the TrueDecoy set, a dataset on which the redocking experiments revealed that KarmaDock and CarsiDock surpassed all physics-based tools in docking accuracy, whereas all physics-based tools notably outperformed AI-based methods in structural rationality. The low physical plausibility of docked structures generated by the top AI method, CarsiDock, mainly stems from insufficient intermolecular validity. The VS results on the TrueDecoy set highlight the effectiveness of RTMScore as a rescore function, and Glide-based methods achieved the highest enrichment factors among all docking tools. Furthermore, we created the RandomDecoy set, a dataset that more closely resembles real-world VS scenarios, where AI-based tools obviously outperformed Glide. Additionally, we found that the employed ligand-based postprocessing methods had a weak or even negative impact on optimizing the conformations of docked complexes and enhancing VS performance. Finally, we proposed a hierarchical VS strategy that could efficiently and accurately enrich active molecules in large-scale VS projects.

You have full access to this article via your institution.

Download PDF Similar content being viewed by others Efficient and accurate large library ligand docking with KarmaDock Article 21 September 2023 An artificial intelligence accelerated virtual screening platform for drug discovery Article Open access 05 September 2024 Network pharmacology and AI in cancer research uncovering biomarkers and therapeutic targets for RALGDS mutations Article Open access 29 March 2025 Main

Molecular docking, a widely used computational technology for studying protein–ligand interactions, has a notable impact on structure-based drug design and discovery1. Typically, this process begins with an experimentally determined or computationally modelled protein structure and aims to predict the binding mode of a specific ligand within the protein pocket and estimate its binding strength using a predefined scoring function (SF)2. This technique is often employed in virtual screening (VS) campaigns, where it docks and ranks a vast library of compounds to help identify potential drug candidates3. Additionally, molecular docking excels in elucidating protein–ligand binding modes, which facilitates the exploration of target functions and the optimization of ligand structures1.

Traditional physics-based docking tools, such as Glide4, LeDock5, rDock6 and Surflex7, typically rely on various search algorithms to explore potential ligand conformations, followed by predefined SFs to rank these pose candidates8. Although these search algorithms can reduce the conformational search space, their accuracy is inherently constrained by the difficulty of achieving full convergence within the limited search steps. More importantly, the SFs used in these docking tools are predominantly physics-based or empirical, suffering from reliability issues due to their simplistic, additive hypotheses9. The rapid advancements in artificial intelligence (AI) offer the potential to revolutionize the design principles of conventional docking programs. One promising direction is the use of deep learning (DL)-based SFs (DLSFs), which leverage the powerful nonlinear fitting capabilities of AI algorithms to capture the intrinsic interactions between ligands and their targets10. These approaches have demonstrated considerable superiority over classical SFs in certain retrospective benchmarks9,11,12,13,14,15,16,17. Another emerging direction is the development of DL-based docking tools (DLDTs) that predict protein–ligand binding poses without depending on heavy sampling, some of which have achieved satisfactory performance in redocking experiments on several public benchmarks18,19,20,21,22,23,24,25. However, they often neglect the physical plausibility of the docked protein–ligand complexes, and their application potential in real-world VS projects remains largely untested. Additionally, the performance of these DL-based tools is highly dependent on the dataset, highlighting the need for rigorous evaluation of their true capabilities26,27,28,29.

Selecting appropriate docking and scoring tools for VS is a fundamental step in enhancing the efficiency of lead compound discovery30. Therefore, it is an urgent need to benchmark various docking and scoring tools to guide the design of VS strategies with improved accuracy and efficiency. To achieve this goal, the following issues must be addressed. First, whereas most current DLDTs claimed to outperform traditional physics-based methods in reproducing the near-native binding poses of crystalized structures, their performance is usually measured by the root-mean-square deviation (RMSD) values between the predicted and true poses18,19,20,21,22,24. There is a lack of emphasis on the physical plausibility of the docked structures, where the investigation would help uncover hidden inductive biases in current AI algorithms and drive the development of more realistic docking protocol31,32. To address the issue, Buttenschoen et al. developed the PoseBusters (PB) package to perform a series of standard quality checks on protein–ligand complexes and found that AI-based docking methods often failed to generate physically valid poses33. With the emergence of more sophisticated docking models, a more in-depth analysis of this issue from more refined perspectives is needed18,20,22,25. Second, most DLDTs have focused primarily on docking accuracy, neglecting their role in the more critical VS task19,21,22,23. Even though some tools have paid attention to their enrichment capability, they often limit their evaluation to datasets consisting solely of artificially created decoys18,20,24, such as DUD-E34 and DEKOIS v.2.0 (ref. 35), where inherent biases might notably impede the reliability of their conclusions36,37. The LIT-PCBA dataset, which collects negative samples from 149 dose-response PubChem bioassays, offers a potential alternative38. However, due to its limitations in target quantity and data quality, the conclusions drawn from such a benchmark remain unconvincing39. Therefore, designing new neutral VS datasets and conducting fair and objective evaluations of current DLDTs are urgently necessary. Finally, as the size of compound libraries continues to grow (for example, ZINC20 (ref. 40), with over 230 million purchasable compounds), balancing the efficiency and accuracy of a VS protocol in a practical VS campaign becomes increasingly important. Making full use of the advantages of different docking tools to create an integrated VS protocol may greatly accelerate drug candidate discovery in large-scale VS projects.

To address the aforementioned issues, we proposed a new benchmark called VSDS-vd (VS datasets with varying decoys), which comprises three datasets with varying decoys: the TrueDecoy set (decoys with truly experimental bioactivity value), the RandomDecoy set (decoys with randomly selected molecules) and the MassiveDecoy set (decoys from a library containing massive compounds). These datasets were designed to comprehensively assess the VS performance of docking tools from different perspectives. Specifically, the TrueDecoy set includes 147 targets for which both active and inactive compounds are experimentally determined, thus mimicking a challenging scenario where high-affinity compounds need to be distinguished from those with low or no affinity. The RandomDecoy set, although inheriting the active compounds from TrueDecoy set, uses molecules randomly selected from a commercial chemical library as decoys, aiming to estimate VS performance in a more realistic setting: that is, enriching active compounds from real-world commercial chemical libraries. The MassiveDecoy set further extends the scale of decoys for certain targets to approximately 7 million, thereby exploring the application potential of the proposed protocol in larger-scale VS campaigns.

Based on our dedicatedly curated datasets, we conducted a systematic assessment of four physics-based docking methods (Glide, Surflex, rDock and LeDock) and four DL-based methods (CarsiDock, KarmaDock, DiffDock and FlexPose). The overall framework of this study and its key components are shown in Fig. 1. Recognizing the importance of docking-pose quality for the reliability of VS results, we initially conducted a redocking experiment on the crystal structures in the TrueDecoy set. This allowed us to assess docking accuracy while considering the structural plausibility of the docking poses. Subsequently, we devised multiple VS protocols by integrating these docking approaches with two state-of-the-art rescoring schemes (RTMScore and EquiScore) and conducted extensive assessments on both the TrueDecoy set and the RandomDecoy set. Finally, based on the VS evaluation results and the characteristics of different DLDTs, we proposed a practical hierarchical strategy by combining different AI-powered tools and validated its performance on the larger-scale MassiveDecoy set.

Fig. 1: Overall framework and key modules in the study.

a, The overall workflow of this work. (Molecule poses which meet crystallographic RMSD within user-defined threshold (commonly 2 Å) and pass all tests in PoseBusters are ‘RMSD and PB-valid’.) b, The processes of constructing VSDS-vd. c,d, Diagrams of the redocking (c) and VS (d) experiments. e, The design philosophy of an integrated protocol for large-scale VS projects.

Full size image Results

To thoroughly evaluate the performance of docking methods and design practical VS protocols in real-world drug screening projects, we present our research results from the following aspects: (1) description of VSDS-vd, (2) assessment of docking capabilities on the TrueDecoy set, (3) VS results for the TrueDecoy and TrueDecoygap sets, (4) VS results for the RandomDecoy set, (5) impact of postprocessing methods on VS results and (6) a hierarchical strategy for practical large-scale VS projects.

Description of VSDS-vd

An ideal dataset for benchmarking different VS protocols should ensure that all included targets are well-studied and possess an appropriate ratio of active to inactive compounds with definite experimental binding data. To achieve this, we initially constructed the TrueDecoy set, where the target information was collected from Uniprot41; ligands and binding data were retrieved from BindingDB42, PubChem43 and ChEMBL44; and the corresponding crystal structures were retrieved from the Protein Data Bank (PDB) database45. All reviewed entries from human sources in BindingDB were checked for target diversity, and an affinity threshold of 1 μM was employed to distinguish between active and inactive compounds. Considering that inactive compounds typically predominate in real-world VS scenarios, we incorporated all identified inactive compounds in the dataset and carefully clustered and selected active compounds to maintain an active-to-inactive ratio of roughly 1:40. As shown in Fig. 2a, the TrueDecoy set ultimately contains a total of 147 targets that cover diverse protein families, including 39 kinases, 26 other enzymes, 23 G protein-coupled receptors (GPCRs), 22 peptidases, 11 nuclear receptors and so on. Figure 2c illustrates that for more than 90% of the targets, the number of active compounds ranges from 20 to 100, with an average of around 38, a maximum of 169 and a minimum of 14. Additionally, to enhance the discriminative ability of the dataset, a subset called TrueDecoygap (the TrueDecoy set with a large bioactivity gap) was also derived from the TrueDecoy set, in which the least-potent active compound is 1,000 times more effective than the most potent inactive compound across all targets. As shown in Supplementary Fig. 1, the TrueDecoygap set contains 71 targets spanning 25 kinases, 16 GPCRs, nine peptidases, eight other enzymes, five nuclear receptors and more.

Fig. 2: Data distribution of the TrueDecoy set and the RandomDecoy set.

a,b, The distribution of target types in the TrueDecoy set (a) and the RandomDecoy set (b). c,d, The distribution of the number of active compounds in the TrueDecoy set (c) and the RandomDecoy set (d).

Source data

Full size image

Acknowledging that compounds below a predetermined bioactivity threshold are not necessarily inactive and still retain the potential for further optimization into drug-like compounds46, the RandomDecoy set was further constructed by introducing compounds randomly selected from a commercial chemical library as decoys, thereby stimulating a more realistic VS scenario. The resulting RandomDecoy set contains 68 targets, where the active compounds for each target are directly retrieved from the TrueDecoy set; the decoys are randomly chosen from the clustered ChemDiv database at a ratio of 1:100, with this selection process repeated three times. The targets still cover a wide range of protein categories, including 20 kinases, 13 peptidases, 11 other enzymes, nine GPCRs, eight nuclear receptors and so on (Fig. 2b), with the number of active compounds following a trend similar to that observed in the TrueDecoy set (Fig. 2d). Additionally, we constructed the MassiveDecoy set, a dataset designed to mimic a practical large-scale VS setting, by fetching all compounds from the Topscience refine set as decoys.

Assessment of docking capability on the TrueDecoy set

We initially assessed the docking capability of four physics-based docking tools (Glide, Surflex, rDock and LeDock) and four DLDTs (CarsiDock, KarmaDock, DiffDock and FlexPose) on the crystal structures of 147 targets in TrueDecoy set, followed by an in-depth analysis of the RMSD values and the physical plausibility of the docked complexes using the PoseBusters package33. To alleviate the influence of potential overlapping with structures used in the model training stage for most DL-based models, we eliminated the PDB entries that were duplicated in the PDBbind database47, leaving 94 structures for additional analysis. Physical plausibility was primarily assessed based on three aspects, including chemical validity and consistency, intramolecular validity and intermolecular validity33.

As shown in Fig. 3a, CarsiDock demonstrates overall superior performance over the other tools, but when a particularly low RMSD threshold (for example, less than 1 Å) is employed to define a successful prediction, the physics-based LeDock notably outperforms the others. A similar performance trend is observed in Fig. 3b, where CarsiDock and LeDock maintain their lead across various RMSD thresholds even after eliminating duplicated target entries. At the conventional threshold of 2 Å (Fig. 3c), CarsiDock achieves the highest success rate of 86.4%, followed by KarmaDock with a success rate of 76.2%. As shown in Fig. 4a, only CarsiDock and KarmaDock correctly predicted the conformation of the ligand in the crystal structure of dipeptidyl peptidase 8 (PDB: 7AYQ) among all docking tools. When the duplicated structures are excluded (Fig. 3d), both CarsiDock and KarmaDock experience a decline in success rate; but CarsiDock retains its leading position, whereas KarmaDock is surpassed by LeDock and Glide. When considering physical plausibility, physics-based methods such as Glide, LeDock and Surflex achieve performance that is superior or comparable to the best DL-based tool, CarsiDock. After excluding specific target entries, the performance of physics-based methods remained relatively stable, whereas the performance of DL-based tools notably decreased, widening the gap between the two types of docking methods. We further explored the sources of physical implausibility in the docked complexes (Fig. 3e,f) and found that almost all complexes generated by physics-based docking tools could pass all physical tests (>90%), whereas notable discrepancies were observed among DL-based tools. Specifically, CarsiDock and DiffDock excel in maintaining intramolecular validity (>90%) but pay little attention to intermolecular conflicts. Unfortunately, FlexPose and KarmaDock perform poorly in nearly all metrics of structural rationality (<80%).

Fig. 3: Redocking results for the TrueDecoy set.

Performance of various docking tools in terms of redocking experiments on the TrueDecoy set. a,b, Cumulative distribution of the RMSD values, where all PDB entries are included (a) or PDB entries duplicated in PDBbind are excluded (b). c,d, Top-1 success rate at the RMSD threshold of 2.0 Å with/without the consideration of the structural rationality of the docked complex, where all PDB entries are included (c) or PDB entries duplicated in PDBbind are excluded (d). e,f, The qualification rate of different metrics for assessing the structural rationality of complexes docked by various tools, where all PDB entries are included (e) or PDB entries duplicated in PDBbind are excluded (f).

Source data

Full size image Fig. 4: Impact of postprocessing methods on DL-based redocking results.

a,b, The docked ligand conformation of dipeptidyl peptidase 8 (PDB: 7AYQ) by various docking tools (a) and the further optimized result of KarmaDock-based docked conformation (b) by the FF and Align methods. c,d, Top-1 success rate at the RMSD threshold of 2 Å with/without consideration of structural rationality on various docking tools when two different methods are incorporated: all cases (c) and removal of duplicated cases in PDBbind (d). e,f, The qualification rate of different metrics for assessing the structural rationality of complexes docked by KarmaDock and postprocessed with FF and Align: all cases (e) and removal of duplicated cases in PDBbind (f).

Source data

Full size image

We further explored the impact of two postprocessing methods, force field optimization (FF) and RDKit-conformation alignment (Align), on enhancing the structural rationality of docked poses in the DL-based redocking experiments. As shown in Fig. 4b, these postprocessing methods notably improve the structural rationality of the ligand conformations generated by KarmaDock on the crystal structure of dipeptidyl peptidase 8 (PDB: 7AYQ), but this improvement comes at the cost of an increased RMSD relative to the crystal ligand: the FF method rises from 1.87 to 1.91 Å, and the Align method escalates to 2.86 Å. As shown in Fig. 4c, different DL-based docking methods exhibit a similar trend in the redocking task. Specifically, the FF method has minimal influence on docking accuracy, whereas the Align method markedly reduces the success rate. When taking physical plausibility into consideration, the indicators for the FF method are slightly improved, whereas those for the Align method are mostly decreased, except for KarmaDock. These findings are valid after excluding duplicated target entries (Fig. 4d). Additionally, as shown in Fig. 4e,f and Supplementary Fig. 2, FF and Align usually enhance the rationality of ligand conformations at the expense of intermolecular metrics. Regarding the physical plausibility of the whole complex, positive effects are only observed on KarmaDock. Overall, postprocessing methods that operate solely on ligand conformation may not yield satisfactory results in optimizing docked poses.

VS results for the TrueDecoy and TrueDecoygap sets

Moreover, we assessed the performance of various VS protocols on the TrueDecoy set, a benchmarking set composed of 147 targets with the bioactivity values of all ligands experimentally determined. An affinity threshold of 1 μM was employed to differentiate between active and inactive compounds, thus making the TrueDecoy set a notably challenging benchmark for VS because its decoys were not entirely non-binders and some of them might own similar scaffolds with active compounds. We conducted VS experiments for both the physics-based and DL-based tools, with RTMScore serving as a rescoring tool to re-estimate the binding strength of the poses produced by the four physics-based approaches and DiffDock. Additionally, the poses generated by Glide were also rescored by EquiScore to give a comparison.

As summarized in Table 1, compared to the previously reported VS results on DUD-E and DEKOIS v.2.0 (ref. 11), the enrichment factors (EFs) of different tools on the TrueDecoy set are significantly lower, highlighting the inherent challenge of this dataset. Among all screening methods, Glide, particularly when combined with RTMScore, achieves the best performance (Supplementary Fig. 3), with the average and median values for EF0.5%, EF1% and EF_5% achieving 6.8727 and 2.0551, 5.6901 and 3.3646, and 3.2980 and 2.3591, respectively. CarsiDock ranks second overall, with corresponding indicators of 5.1080 and 0, 4.8480 and 1.7354, and 3.1694 and 2.1988, respectively. For physics-based docking tools, the use of RTMScore for rescoring in most cases produces better results than their original SFs. Furthermore, it is evident that although EquiScore claims to remarkably improve the VS performance of Glide on DEKOIS v.2.0, it performs poorly on the TrueDecoy set, further highlighting the dataset’s inherent challenges.

Table 1 Performance of various VS protocols on the TrueDecoy set and TrueDecoygap set Full size table

To further enhance the discriminative capability of the benchmark, we also tested these protocols on the TrueDecoygap set (Table 1 and Supplementary Fig. 3), a derivative dataset that has a wider bioactivity gap between the active and decoy compounds. As expected, the indicators observed here are generally higher than those on the TrueDecoy set, but our earlier findings remain consistent.

VS results for the RandomDecoy set

To assess the docking methods in a more realistic VS scenario, we further tested the above protocols on the RandomDecoy set, a dataset that consists of 68 targets; the active compounds remained consistent with those in the TrueDecoy set, but the decoy molecules were randomly selected from the clustered ChemDiv database at a 1:100 ratio, repeated three times. Given the notably larger number of compounds in the RandomDecoy set compared to the TrueDecoy set, we excluded several poorly performing protocols from the TrueDecoy set for this testing phase. Balancing both efficiency and accuracy, we retained five representative protocols for testing: CarsiDock, KarmaDock, Glide and Glide rescored by either RTMScore or EquiScore.

As shown in Fig. 5a,b, regardless of whether the active-to-decoy ratio is 1:100 or 1:300, the overall rankings for the EFs on the RandomDecoy set remain consistent: CarsiDock, KarmaDock, Glide combined with RTMScore, Glide and Glide combined with EquiScore. Figure 5c,d further illustrate that in terms of the average values of the EF0.5%, EF%1 and EF_%5 metrics on the RandomDecoy set (1:100) and the RandomDecoy set (1:300), CarsiDock achieved the highest performance, with EFs of 59.3, 43.5 and 12.6, and 81.0, 47.6 and 12.8, respectively, followed by KarmaDock. Rescoring Glide-based docked complexes with RTMScore also led to an improvement in the average values of these three metrics from 40.8, 28.8, 8.8 to 51.0, 34.4, 9.9 and 51.7, 30.4, 8.9 to 65.4, 37.1, 10.0 on the RandomDecoy set (1:100) and the RandomDecoy set (1:300), respectively. These results highlight the potential of DL-based docking and scoring tools in real-world drug screening processes.

Fig. 5: VS results for the RandomDecoy and MassiveDecoy sets.

a–d, Distribution (a and b; number of data points n = 68) and average (c and d) of EF0.5%, EF1% and EF_5% achieved by various VS protocols on the RandomDecoy set, with the active-to-decoy ratio set to 1:100 (a and c; number of biological replicates n = 3) and 1:300 (b and d), respectively. c, The three black dots on top of each bar represent the EF value calculated from random selections of decoys three times. e, The overall workflow of the hierarchical strategy with KarmaDock and CarsiDock. f–h, The number of active compounds enriched by KarmaDock and KarmaDock + CarsiDock among the top 50 (f), 100 (g) and 200 (h) compounds. a,b Boxplots start with the median (Q2, 50th percentile) as the centreline. Each successive level outwards contains half the remaining data. This continues until arriving at the outlier level. Outliers outside the outlier whiskers are shown as distinct points.The method for calculating outliers is to end up with around 5–8 outliers in each tail.

Source data

Full size image Impact of postprocessing methods on VS results

The fundamental approach to evaluating the effectiveness of postprocessing techniques is to ascertain whether they could enhance the efficiency of identifying active compounds in VS projects. Following the evaluation results of various postprocessing methods in redocking experiments, we conducted further investigations through FF and Align optimization on KarmaDock-based docked structures across diverse benchmarking sets. The results are listed in Supplementary Table 1. On both benchmarking sets, the FF optimization method shows a nearly negligible and inconsistently uniform impact on the average and median values of different EFs. For the Align optimization method, the EFs demonstrate a clear and consistent declining trend on both benchmarks. This finding aligns with its behaviour observed in the redocking experiments. Specifically, the FF optimization method had a relatively modest effect on ligand conformation, resulting in minimal changes to the scoring hierarchy among different ligands. In contrast, whereas the Align optimization method substantially improved the rationality of ligand conformations, it compromised the precision of ligand coordinates, severely impacting the scoring of the docked complexes. Overall, these postprocessing methods failed to enhance VS performance, highlighting the need for researchers to focus on developing more precise docking tools directly.

A hierarchical strategy for practical large-scale VS projects

One of the most important applications of docking and scoring tools is to identify potential active compounds from extensive compound libraries through VS approaches. With the ever-increasing size of compound libraries, it is imperative to balance both efficiency and accuracy when designing suitable docking tools for real-world VS projects. As previously demonstrated, CarsiDock achieved the highest EFs on the RandomDecoy set, but its efficiency is limited, as docking a ligand to its target takes 1.726 s, making it impractical to screen libraries containing over a million compounds18,48. Although KarmaDock is less effective than CarsiDock in enriching ligands, it completes a single docking experiment in just 0.017 s, allowing it to process a million molecules in merely 4.7 h on a single Tesla V100 (ref. 20). Therefore, as shown in Fig. 5e, we proposed a hierarchical VS strategy that leverages the respective strengths of both tools. Specifically, KarmaDock is initially used for preliminary rapid screening of the large-scale compound library. Subsequently, the top 100,000 ranked molecules are docked and scored further using CarsiDock to obtain the final ranking.

To evaluate the performance of the hierarchical strategy in practical VS projects, we selected eight targets spanning various protein families, including Kinase, Phosphatase, GPCR, Peptidase and Ligase, and gathered all their active compounds. The ligands for each target were clustered, and only one representative ligand from each cluster was selected to form a practical large-scale VS dataset, combined with the TopScience Refine Set, a commercial chemical library with more than 7 million compounds. We performed screening experiments on this dataset using both the hierarchical strategy and KarmaDock alone, with the results shown in Fig. 5f–h. As is evident, the hierarchical strategy generally outperforms the standalone use of KarmaDock in identifying a greater number of active molecules within the top 50 ranked compounds, except for the targets Serine/threonine-protein kinase PLK1 (Uniprot: P53350) and Tyrosine-protein phosphatase non-receptor type 1 (Uniprot: P18013). Notably, KarmaDock struggled to identify any active molecules in the top 200 compounds for Cholinesterase, Adenosine receptor A2a and Collagenase 3 (Uniprot: P06276, P29274 and P45452), whereas the hierarchical strategy successfully retrieved 12, 38 and six active molecules, respectively. For the phosphatidylinositol 4,5-bisphosphate 3-kinase catalytic subunit delta isoform (Uniprot: O00329), the hierarchical strategy shows notably improved ability to enrich ligands, increasing the number of active compounds in the top 50, 100 and 200 from 23, 31 and 44 to 39, 71 and 120, respectively. Furthermore, the results indicate that the efficiency of this strategy in retrieving active molecules varies across different targets, suggesting a degree of target dependence. In conclusion, the hierarchical VS strategy offers a promising approach for efficient and accurate screening of active compounds from large-scale compound libraries.

Discussion

With the rapid advancement of AI technology, various DL-based tools for protein–ligand docking and scoring have emerged, demonstrating remarkable potential in computational efficiency and accuracy. However, these tools often overlook the physical plausibility of docked protein–ligand complexes and their practical applicability in real-world VS projects, which plays an essential role in identifying potential active compounds and elucidating target-drug binding patterns. To enhance drug discovery efforts, this study benchmarked four AI-powered docking tools, four physics-based tools and two AI-powered rescoring methods using various datasets. Based on the evaluation results for the different datasets, this study further proposed a purely AI-powered hierarchical strategy for practical large-scale VS projects.

Considering that the quality of docking poses is fundamental to the reliability of VS results, we initially redocked the ligand within the holo structure of targets in the TrueDecoy set using a variety of docking tools. These results indicate that DL-based KarmaDock and CarsiDock surpassed all physics-based tools on RMSD, whereas all physics-based tools outperformed AI-based methods on the physical plausibility of the docked complexes. When evaluating both criteria, CarsiDock, the top AI-powered tool, lagged behind physics-based tools. The trend is particularly evident for KarmaDock, which nearly fails to generate any valid complex poses, primarily due to the fact that it directly updates the atomic coordinates of the ligand while not considering the bond parameters. CarsiDock, in addition to learning the protein–ligand atom distance matrices, also introduces a translation, rotation and torsion-guided geometry optimization procedure to reconstruct the matrices into a credible binding pose. Notably, the low physical plausibility of structures docked by CarsiDock was primarily attributed to intermolecular metrics, with intramolecular validation accuracy exceeding 90%. This phenomenon reminds us to pay more attention to the geometric distance between atoms in the ligand and protein when designing future DL-based docking algorithms. Furthermore, upon excluding the PDB entries already present in the PDBbind dataset, the redocking performance of DL-based tools markedly declined across different metrics, whereas physics-based methods showed almost no change, which highlights the data dependency of DL-based tools.

The decoy molecules in the TrueDecoy set are experimentally determined, and despite their relatively low bioactivity, most can bind to the target to some extent and often share similar scaffolds with active compounds. Consequently, the TrueDecoy set poses a more challenging benchmarking set for VS compared to the RandomDecoy set, where decoy molecules are randomly chosen from compound libraries. Glide, either alone or combined with RTMScore for rescoring, achieved the highest EFs on the TrueDecoy set, and CarsiDock and KarmaDock outperformed Glide on the RandomDecoy set. This highlights the importance of researchers selecting docking tools flexibly based on the specific application to accelerate lead compound discovery. CarsiDock and KarmaDock may be well-suited for enriching active compounds from large-scale compound libraries, whereas Glide may be more effective when narrowing down potential active compounds to identify the most potent ones.

Two postprocessing methods (FF and Align) were employed to refine the conformations generated by DL-based tools. Redocking experiments revealed that although these methods enhanced the structural rationality of ligand conformations, they often compromised the physical plausibility of the entire protein–ligand complex. The only exception is KarmaDock, prompting its use for further VS on both the TrueDecoy set and the RandomDecoy set with these two postprocessing methods. The results on both datasets demonstrate that the FF method has a minor and variable impact on VS outcomes, whereas the Align method consistently and notably diminished VS performance. Therefore, future research in DL-based tools may focus on developing more precise docking algorithms or designing postprocessing methods that optimize the entire complex structure.

It is typically necessary to balance speed and accuracy when choose docking tools for practical VS projects. KarmaDock stands out for its speed, whereas CarsiDock offers exceptional accuracy. To harness the strengths of both tools, we proposed a hierarchical strategy: first, KarmaDock is utilized for initial screening of the entire compound library; subsequently, the top 100,000 molecules are ranked using CarsiDock. This methodology has proven successful in VS across eight targets within a library of over 7 million compounds, highlighting its potential in real-world drug screening.

In general, a balanced perspective is essential when evaluating DLDTs. On one hand, they have achieved VS performance comparable to or surpassing traditional physics-based methods in certain screening scenarios, and some of them are exponentially more efficient, enabling them to explore broader chemical spaces and increasing the probability of discovering potential new drugs with scaffolds. On the other hand, DL-based tools have their limitations. The lack of physical plausibility in their docked structures raises concerns about the reliability of screening results. As purely data-driven tools, they exhibit variability in docking and scoring across different targets, complicating the assessment of their suitability for specific target proteins and limiting their widespread application. In more demanding scenarios, such as distinguishing high-activity compounds from low-activity ones, DL-based tools may be less effective than physics-based methods. In conclusion, to efficiently and accurately identify potential active compounds and accelerate drug discovery, we should flexibly select various docking tools that leverage their unique strengths in practical drug screening processes.

Methods Dataset curation and preparation

In this study, we created three datasets—the TrueDecoy set, the RandomDecoy set and the MassiveDecoy set—to benchmark various VS protocols. Their major difference lies in their methods of generating negative samples. Specifically, the negative samples in the TrueDecoy set are all experimentally validated low-activity or inactive compounds, whereas those in the RandomDecoy set and the MassiveDecoy set are decoys selected from commercial chemical libraries with different rules, thus simulating a variety of VS scenarios.

To construct the TrueDecoy set, we collected all target information from BindingDB42, retaining only those annotated as human targets in the Uniprot database41. Subsequently, we collected the ligands and their bioactivity data for these targets from the BindingDB42, PubChem43 and ChEMBL44 databases. A threshold value of 1 μM was defined to classify molecules based on their experimental bioactivity data with different units, including IC50 (half-maximal inhibitory concentration), Ki (inhibition constant) and Kd (dissociation constant), categorizing those below the threshold as active and those above as inactive, and the compounds with vague labels were removed49. If a molecule had multiple activity values reported in the database, and the different values resulted in the same classification label, it was retained. Otherwise, molecules with ambiguous labels were discarded. To maintain an active-to-inactive compound ratio of approximately 1:40, targets with fewer than 800 inactive compounds were eliminated. For the remaining targets, we utilized PDBminer50 to retrieve the complete PDB entries of their crystal structures and discarded entries with holo structures. In cases where multiple holo structures were present, we tended to select the one with the highest resolution. In addition, we attempted to dock all active molecules of one target into each experimental structure of that target using Glide and calculated the proportion of active molecules that could successfully dock into the binding pocket of each structure. We ensured that the receptor structures selected for docking could successfully accommodate more than 90% of the active molecules. This process yielded a total of 147 targets representing a broad spectrum of protein families. Detailed information about the TrueDecoy set can be found in Supplementary Table 2.

Considering that the active compounds for a specific target might share structural similarities, we followed the construction workflow from DUD-E and clustered the active compounds for each target based on Bemis–Murcko scaffolds34. The inactive compounds for each target were directly used as decoy molecules in the TrueDecoy set. The necessary number of active compounds for the TrueDecoy set was determined by the inactive compound count and the predefined 1:40 active-to-inactive ratio. When the number of clusters exceeded the required quantity, we selected the most bioactive one from each cluster, further ranked them based on their bioactivity and then selected the top-ranked compounds to meet the required number. Otherwise, if the number of clusters was insufficient, we increased the number of selected compounds from each cluster until the requirement was satisfied. To standardize the Simplified Molecular Input Line Entry System representation of all compounds, we processed them using the Cleanup, FragmentParent, Uncharger and Tautomer Enumerate functions in the rdMolStandardize module of the RDKit cheminformatics toolkit. For compounds consisting of multiple components, only the largest component was retained.

To be precise, the TrueDecoy set is designed to evaluate the ability of different docking tools to enrich high-activity molecules from a pool of relatively low-activity or inactive molecules. However, using a uniform threshold of 1 µM across all units may lead to a scenario where a decoy molecule binds more strongly to the target than an active molecule. To address this issue, we designed an additional dataset, the TrueDecoygap set, where the minimum bioactivity of active compounds was 1,000 times higher than the maximum bioactivity of inactive ones for all targets. The likelihood of decoy molecules in the TrueDecoygap set binding more tightly than active molecules is extremely low, thus making it a more objective benchmark for docking protocols comparison. The TrueDecoygap set contains 71 targets spanning various protein families. Detailed information on the TrueDecoygap set is listed in Supplementary Table 3.

The TrueDecoy set fails to fully replicate the real-world VS scenario because its decoy molecules, defined by a predetermined bioactivity threshold (that is, 1 μM), theoretically still hold the potential for optimization into drug-like compounds. To address this, we designed the RandomDecoy set, with 68 targets selected from different protein families in the TrueDecoy set. Although the active compounds for each target remain consistent with the TrueDecoy set, the decoy molecules were randomly selected three times from the clustered ChemDiv database at a 1:100 ratio. Detailed information on the RandomDecoy set is listed in Supplementary Table 4. Although the RandomDecoy set mimics a more realistic VS setting, it still doesn’t fully represent an actual VS scenario due to the notably lower proportion of active molecules compared to real-world drug screening projects, which typically use ratios far exceeding 1:100. Hence, we further constructed the MassiveDecoy set, which could be considered a practical VS benchmark. Its construction involved the selection of eight targets from diverse protein families, clustering their active compounds and mixing one compound from each cluster with the Topscience refine set, a commercial compound library containing 7,172,268 compounds. More information about the MassiveDecoy set can be found in Supplementary Table 5.

The crystal structures used in this study were retrieved from the PDB45 database and prepared using the default settings of the Protein Preparation Wizard module in the Schrödinger 2018 suite51, including adding hydrogens, assigning bond orders, filling missing side-chain atoms and loops, assigning the protein protonation state at pH = 7.4 using PROPKA52, optimizing the hydrogen bond network and minimizing the entire structure until the RMSD of heavy atoms averaged 0.30 Å. Then, thorough inspections were performed to ensure that the prepared structures could be compatible with the RDKit toolkit.

Docking and scoring methods

In this study, we benchmarked a total of four DLDTs (CarsiDock18, KarmaDock20, DiffDock19 and FlexPose22), four physics-based docking methods (Glide4, LeDock5, rDock6 and Surflex7) and two DLSFs (RTMScore11 and EquiScore13).

The four selected DLDTs have different design principles. Briefly, CarsiDock employs a DL model to forecast protein–ligand atomic-distance matrices, followed by a geometry optimization procedure to convert these matrices into a reliable binding pose, with RTMScore providing the final binding affinity18. KarmaDock uses E(n) equivariant graph neural networks with self-attention for ligand pose prediction and employs a statistical potential derived from mixture density network to estimate protein–ligand binding strength20. FlexPose, a geometric graph network, flexibly models complex structures in Euclidean space and provides an affinity estimation module for binding affinity prediction22. DiffDock utilizes equivariant graph neural networks within a diffusion process for ligand pose generation, but it does not have its own SF for binding strength estimation19. Among these four methods, DiffDock performs blind docking, whereas the other three need predetermined binding pockets. As for the two DLSFs, RTMScore learns a statistical potential based on the distance likelihood between ligand heavy atoms and binding pocket residues11, whereas EquiScore is a heterogeneous graph neural network that integrates physical prior knowledge and characterizes protein–ligand interactions in equivariant geometric space13.

DL-based approaches were executed according to their official instructions. For physics-based docking approaches, unless otherwise specified, all parameters were set to default values, as detailed below.

Glide

Glide employs a docking grid to improve processing efficiency. The grid box was positioned at the centre of mass of the reference ligand within the holo structure. The size of the grid box was determined as 16 Å plus 0.8 times the ligand’s diameter, representing the maximum distance between any two atoms in the cocrystalized ligand. Glide with standard precision was used to calculate the docking scores.

LeDock

Binding poses were sampled using a combination of simulated annealing and evolutionary optimization technologies and then scored by the default SF.

Surflex

The pgeom mode was activated with the multistart parameter set to 10. Docking scores were calculated using the default SF.

rDock

A mix of stochastic and deterministic methods were employed for pose sampling. The default SF was utilized for affinity estimation.

In redocking experiments, only the top-ranked poses produced by different docking methods were analysed. For VS tests, a maximum of ten poses were generated per complex by the four physics-based docking tools. Besides using their respective SFs, all poses generated by each program were further reranked using RTMScore. For Glide, poses were also rescored by EquiScore because Glide together with EquiScore yielded the best performance in the original publication. It meant that the top-ranked pose selected by RTMScore/EquiScore was employed as the final binding pose for a specific protein–ligand complex. Due to the absence of a built-in SF, the ten poses generated by DiffDock were rescored by RTMScore. For the other three DLDTs, their internal SFs were employed.

Postprocessing methods for DL-based docking

Two postprocessing methods, FF and Align, are used here to explore their role in redocking experiments and VS tasks. The FF optimization can be easily implemented by MMFFOptimizeMolecule, an application programming interface of RDKit, which uses the MMFF94 Force Field to optimize the predicted ligand conformations with a maximum of ten steps. The Align optimization involves aligning the chemically plausible RDKit conformations, which lack accurate orientation and torsional angles, with the predicted binding poses and replacing the initially predicted poses with the appropriately aligned RDKit conformations. This process involves initially defining the ligands’ torsional angles and assigning them to the RDKit conformations, followed by aligning these modified conformations with the predicted binding poses using the Kabsch algorithm, ultimately replacing the initial predictions with the more accurately aligned structures.

Evaluation metrics

In the redocking experiments, we analysed the docking accuracy and physical plausibility of the top-ranked conformation generated by different docking methods. The docking accuracy was measured by the success rate of generating binding conformations with an RMSD of less than 2 Å compared to the ground-truth conformations. The physical plausibility of predicted conformations was analysed by PoseBusters, a Python package that utilizes the well-established cheminformatics toolkit RDKit to conduct a series of standard quality assessments on docked conformations33. The assessment mainly includes three aspects: the chemical validity and consistency of the generated ligand conformation, the intramolecular validity of the generated ligand conformation and the intermolecular validity of the complex conformation—specifically, the chemical validity and consistency emphasis on ligand sanitization, tetrahedral chirality, double bond stereochemistry and so on. The intramolecular validity centres around the structural rationality of the ligand, including bond lengths, bond angles, internal steric clash, energy ratio and so on. The intermolecular validity mainly focuses on protein–ligand clashes. Only complexes that pass all these checks can be considered structurally rational. More details about PoseBusters can be found in the original publication33. In the VS tasks, the enriching power of different protocols is mainly measured by EFs at thresholds of 0.5%, 1% and 5%, calculated by the oddt package (v.0.7)53.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Data availability

The raw dataset is collected from https://www.bindingdb.org/rwd/bind/index.jsp (ref. 42), https://www.ebi.ac.uk/chembl/ (ref. 44) and https://pubchem.ncbi.nlm.nih.gov/ (ref. 43). The prepared benchmarking set is available at https://doi.org/10.5281/zenodo.13684010 (ref. 54). All structures are available in PDB (https://www.rcsb.org/)45. All target information can be found at https://www.uniprot.org/ (ref. 41). Source data are provided with this paper.

Code availability

Our code to compare different methods is available via GitHub at https://github.com/shukai1997/VSDS-VD and via Zenodo at https://doi.org/10.5281/zenodo.14649209 (ref. 55). The codes for implementing the different models are available via GitHub at https://github.com/carbonsilicon-ai/CarsiDock, https://github.com/schrojunzhang/KarmaDock, https://github.com/tiejundong/FlexPose, https://github.com/gcorso/DiffDock, https://github.com/sc8668/RTMScore and https://github.com/CAODH/EquiScore.

References

Meng, X. Y., Zhang, H. X., Mezei, M. & Cui, M. Molecular docking: a powerful approach for structure-based drug discovery. Curr. Comput. Aided Drug Des. 7, 146–157 (2011).

Article   MATH   Google Scholar

Forli, S. et al. Computational protein-ligand docking and virtual drug screening with the AutoDock suite. Nat. Protoc. 11, 905–919 (2016).

Article   MATH   Google Scholar

Lionta, E., Spyrou, G., Vassilatis, D. K. & Cournia, Z. Structure-based virtual screening for drug discovery: principles, applications and recent advances. Curr. Top. Med. Chem. 14, 1923–1938 (2014).

Article   Google Scholar

Friesner, R. A. et al. Glide: a new approach for rapid, accurate docking and scoring. 1. Method and assessment of docking accuracy. J. Med. Chem. 47, 1739–1749 (2004).

Article   MATH   Google Scholar

Zhao, H., Huang, D. & Caflisch, A. Discovery of tyrosine kinase inhibitors by docking into an inactive kinase conformation generated by molecular dynamics. ChemMedChem 7, 1983–1990 (2012).

Article   Google Scholar

Ruiz-Carmona, S. et al. rDock: a fast, versatile and open source program for docking ligands to proteins and nucleic acids. PLoS Comput. Biol. 10, e1003571 (2014).

Article   Google Scholar

Jain, A. N. Surflex: fully automatic flexible molecular docking using a molecular similarity-based search engine. J. Med. Chem. 46, 499–511 (2003).

Article   MATH   Google Scholar

Pagadala, N. S., Syed, K. & Tuszynski, J. Software for molecular docking: a review. Biophys. Rev. 9, 91–102 (2017).

Article   Google Scholar

Wang, Z. et al. Comprehensive evaluation of ten docking programs on a diverse set of protein-ligand complexes: the prediction accuracy of sampling power and scoring power. Phys. Chem. Chem. Phys. 18, 12964–12975 (2016).

Article   Google Scholar

Yu, Y., Lu, S., Gao, Z., Zheng, H. & Ke, G. Proc. ICLR 2023—Machine Learning for Drug Discovery Workshop (ICLR, 2023).

Shen, C. et al. Boosting protein-ligand binding pose prediction and virtual screening based on residue-atom distance likelihood potential and graph transformer. J. Med. Chem. 65, 10691–10706 (2022).

Article   MATH   Google Scholar

Shen, C. et al. A generalized protein-ligand scoring framework with balanced scoring, docking, ranking and screening powers. Chem. Sci. 14, 8129–8146 (2023).

Article   MATH   Google Scholar

Cao, D. H. et al. Generic protein-ligand interaction scoring by integrating physical prior knowledge and data augmentation modelling. Nat. Mach. Intell. 6, 688–700 (2024).

Article   MATH   Google Scholar

Moon, S., Zhung, W., Yang, S., Lim, J. & Kim, W. Y. PIGNet: a physics-informed deep learning model toward generalized drug-target interaction predictions. Chem. Sci. 13, 3661–3673 (2022).

Article   Google Scholar

Moon, S., Hwang, S. Y., Lim, J. & Kim, W. Y. PIGNet2: a versatile deep learning-based protein-ligand interaction prediction model for binding affinity scoring and virtual screening. Digit. Discov. 3, 287–299 (2024).

Article   MATH   Google Scholar

Wojcikowski, M., Ballester, P. J. & Siedlecki, P. Performance of machine-learning scoring functions in structure-based virtual screening. Sci. Rep. 7, 46710 (2017).

Article   MATH   Google Scholar

Kwon, Y., Shin, W. H., Ko, J. & Lee, J. AK-Score: accurate protein-ligand binding affinity prediction using an ensemble of 3D-convolutional neural networks. Int. J. Mol. Sci. 21, 8424 (2020).

Article   MATH   Google Scholar

Cai, H. et al. CarsiDock: a deep learning paradigm for accurate protein-ligand docking and screening based on large-scale pre-training. Chem. Sci. 15, 1449–1471 (2024).

Article   MATH   Google Scholar

Corso, G., Stärk, H., Jing, B., Barzilay, R. & Jaakkola, T. S. Proc. Eleventh International Conference on Learning Representations (ICLR, 2023).

Zhang, X. et al. Efficient and accurate large library ligand docking with KarmaDock. Nat. Comput. Sci. 3, 789–804 (2023).

Article   MATH   Google Scholar

Li, Y., Li, L., Wang, S. & Tang, X. EQUIBIND: a geometric deep learning-based protein-ligand binding prediction method. Drug Discov. Ther. 17, 363–364 (2023).

Article   MATH   Google Scholar

Dong, T., Yang, Z., Zhou, J. & Chen, C. Y. Equivariant flexible modeling of the protein-ligand binding pose with geometric deep learning. J. Chem. Theory Comput. 19, 8446–8459 (2023).

Article   MATH   Google Scholar

Lu, W. et al. TANKBind: trigonometry-aware neural networks for drug-protein binding structure prediction. In Advances in Neural Information Processing Systems 35 (eds Koyejo, S. et al.) 7236–7249 (NeurIPS, 2022).

Geng, M. et al. Proc. Eleventh International Conference on Learning Representations (ICLR, 2023).

Zhang, X. J. et al. Advancing ligand docking through deep learning: challenges and prospects in virtual screening. Acc. Chem. Res. 57, 1500–1509 (2024).

Article   MATH   Google Scholar

Chatterjee, A. et al. Improving the generalizability of protein-ligand binding predictions with AI-Bind. Nat. Commun. 14, 1989 (2023).

Article   MATH   Google Scholar

Sieg, J., Flachsenberg, F. & Rarey, M. In need of bias control: evaluating chemical data for machine learning in structure-based virtual screening. J. Chem. Inf. Model. 59, 947–961 (2019).

Article   MATH   Google Scholar

Bai, P. Z., Miljkovic, F., John, B. & Lu, H. P. Interpretable bilinear attention network with domain adaptation improves drug-target prediction. Nat. Mach. Intell. 5, 126–136 (2023).

Article   MATH   Google Scholar

Siebenmorgen, T. et al. MISATO: machine learning dataset of protein-ligand complexes for structure-based drug discovery. Nat. Comput. Sci. 4, 367–378 (2024).

Article   MATH   Google Scholar

Agu, P. C. et al. Molecular docking as a tool for the discovery of molecular targets of nutraceuticals in diseases management. Sci. Rep. 13, 13398 (2023).

Article   MATH   Google Scholar

Hawkins, P. C., Skillman, A. G., Warren, G. L., Ellingson, B. A. & Stahl, M. T. Conformer generation with OMEGA: algorithm and validation using high quality structures from the Protein Databank and Cambridge Structural Database. J. Chem. Inf. Model. 50, 572–584 (2010).

Article   Google Scholar

Friedrich, N. O. et al. Benchmarking commercial conformer ensemble generators. J. Chem. Inf. Model. 57, 2719–2728 (2017).

Article   MATH   Google Scholar

Buttenschoen, M., Morris, G. M. & Deane, C. M. PoseBusters: AI-based docking methods fail to generate physically valid poses or generalise to novel sequences. Chem. Sci. 15, 3130–3139 (2024).

Article   Google Scholar

Mysinger, M. M., Carchia, M., Irwin, J. J. & Shoichet, B. K. Directory of useful decoys, enhanced (DUD-E): better ligands and decoys for better benchmarking. J. Med. Chem. 55, 6582–6594 (2012).

Article   Google Scholar

Bauer, M. R., Ibrahim, T. M., Vogel, S. M. & Boeckler, F. M. Evaluation and optimization of virtual screening workflows with DEKOIS 2.0-a public library of challenging docking benchmark sets. J. Chem. Inf. Model. 53, 1447–1462 (2013).

Article   Google Scholar

Chen, L. et al. Hidden bias in the DUD-E dataset leads to misleading performance of deep learning in structure-based virtual screening. PLoS ONE 14, e0220113 (2019).

Article   Google Scholar

Stein, R. M. et al. Property-unmatched decoys in docking benchmarks. J. Chem. Inf. Model. 61, 699–714 (2021).

Article   MATH   Google Scholar

Tran-Nguyen, V. K., Jacquemard, C. & Rognan, D. LIT-PCBA: an unbiased data set for machine learning and virtual screening. J. Chem. Inf. Model. 60, 4263–4273 (2020).

Article   Google Scholar

Rohrer, S. G. & Baumann, K. Maximum unbiased validation (MUV) data sets for virtual screening based on PubChem bioactivity data. J. Chem. Inf. Model. 49, 169–184 (2009).

Article   MATH   Google Scholar

Irwin, J. J. et al. ZINC20—a free ultralarge-scale chemical database for ligand discovery. J. Chem. Inf. Model. 60, 6065–6073 (2020).

Article   MATH   Google Scholar

UniProt C. UniProt: the universal protein knowledgebase in 2023. Nucleic Acids Res. 51, D523–D531 (2023).

Article   Google Scholar

Liu, T., Lin, Y., Wen, X., Jorissen, R. N. & Gilson, M. K. BindingDB: a web-accessible database of experimentally determined protein-ligand binding affinities. Nucleic Acids Res. 35, D198–D201 (2007).

Article   Google Scholar

Kim, S. et al. PubChem 2023 update. Nucleic Acids Res. 51, D1373–D1380 (2023).

Zdrazil, B. et al. The ChEMBL database in 2023: a drug discovery platform spanning multiple bioactivity data types and time periods. Nucleic Acids Res. 52, D1180–D1192 (2024).

Article   Google Scholar

Rose, P. W. et al. The RCSB Protein Data Bank: new resources for research and education. Nucleic Acids Res. 41, D475–D482 (2013).

Article   MATH   Google Scholar

Chen, S. et al. Identification of 3-aryl-5-methyl-isoxazole-4-carboxamide derivatives and analogs as novel HIF-2alpha agonists through docking-based virtual screening and structural modification. Eur. J. Med. Chem. 268, 116227 (2024).

Article   Google Scholar

Wang, R., Fang, X., Lu, Y. & Wang, S. The PDBbind database: collection of binding affinities for protein-ligand complexes with known three-dimensional structures. J. Med. Chem. 47, 2977–2980 (2004).

Article   MATH   Google Scholar

Shen, C. et al. DrugFlow: an AI-driven one-stop platform for innovative drug discovery. J. Chem. Inf. Model. 64, 5381–5391 (2024).

Article   MATH   Google Scholar

Gu, S. K. et al. AMGC is a multiple-task graph neutral network for epigenetic target profiling. Cell Rep. Phys. Sci. 5, 101850 (2024).

Article   MATH   Google Scholar

Degn, K., Beltrame, L., Tiberti, M. & Papaleo, E. PDBminer to find and annotate protein structures for computational analysis. J. Chem. Inf. Model. 63, 7274–7281 (2023).

Article   MATH   Google Scholar

Sastry, G. M., Adzhigirey, M., Day, T., Annabhimoju, R. & Sherman, W. Protein and ligand preparation: parameters, protocols, and influence on virtual screening enrichments. J. Comput. Aided Mol. Des. 27, 221–234 (2013).

Article   Google Scholar

Olsson, M. H., Sondergaard, C. R., Rostkowski, M. & Jensen, J. H. PROPKA3: consistent treatment of internal and surface residues in empirical pKa predictions. J. Chem. Theory Comput. 7, 525–537 (2011).

Article   MATH   Google Scholar

Wojcikowski, M., Zielenkiewicz, P. & Siedlecki, P. Open Drug Discovery Toolkit (ODDT): a new open-source player in the drug discovery field. J. Cheminform. 7, 26 (2015).

Article   MATH   Google Scholar

Gu, S. VSDS-VD: Benchmarking AI-powered docking methods from the perspective of virtual screening. Zenodo https://doi.org/10.5281/zenodo.13684010 (2024).

Gu, S. Code for ‘Benchmarking AI-powered docking methods from the perspective of virtual screening’. Zenodo https://doi.org/10.5281/zenodo.14649209 (2024).

Download references

Acknowledgements

This work was financially supported by the National Natural Science Foundation of China (grant nos. 92370130, 22220102001 and 22303081) and the National Key Research and Development Programme of China (grant no. 2022YFF1203003).

Author information Author notes

These authors contributed equally: Shukai Gu, Chao Shen, Xujun Zhang.

Authors and Affiliations

College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, China

Shukai Gu, Xujun Zhang, Huifeng Zhao, Hongyan Du, Yihao Zhao, Tingjun Hou & Yu Kang

Faculty of Applied Sciences, Macao Polytechnic University, Macao, China

Shukai Gu, Bo Liu, Chenggong Fu, Silong Zhai & Huanxiang Liu

Department of Clinical Pharmacy, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China

Chao Shen

Department of Medicinal Chemistry, China Pharmaceutical University, Nanjing, China

Huiyong Sun

CarbonSilicon AI Technology Co. Ltd, Hangzhou, China

Heng Cai, Hao Luo & Yafeng Deng

Contributions

Y.K., T.H. and S.G. designed the research study. S.G. constructed the dataset and wrote the code. S.G., C.S., X.Z., H.S., H.C., H. Luo, H.Z. B.L., H.D. and Y.Z. performed the analysis. S.G., C.F., S.Z., Y.D. and H. Liu wrote the paper. All authors read and approved the paper.

Corresponding authors

Correspondence to Tingjun Hou or Yu Kang.

Ethics declarations Competing interests

The authors declare no competing interests.

Peer review Peer review information

Nature Machine Intelligence thanks Claudio N. Cavasotto, Mingyue Zheng and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information Supplementary Information

Supplementary Figs. 1–3 and Tables 1–5.

Reporting Summary Supplementary Data 1

Data distribution of the TrueDecoygap set.

Supplementary Data 2

Impact of postprocessing methods on DL-based redocking results.

Supplementary Data 3

Results of various VS protocols on the TrueDecoy and TrueDecoygap sets.

Source data Source Data Fig. 2

Statistical source data.

Source Data Fig. 3

Statistical source data.

Source Data Fig. 4

Statistical source data.

Source Data Fig. 5

Statistical source data.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article Cite this article

Gu, S., Shen, C., Zhang, X. et al. Benchmarking AI-powered docking methods from the perspective of virtual screening. Nat Mach Intell 7, 509–520 (2025). https://doi.org/10.1038/s42256-025-00993-0

Download citation

Received 24 September 2024

Accepted 15 January 2025

Published 13 February 2025

Version of record 13 February 2025

Issue date March 2025

DOI https://doi.org/10.1038/s42256-025-00993-0

Share this article

Anyone you share the following link with will be able to read this content:

Get shareable link

Provided by the Springer Nature SharedIt content-sharing initiative

Subjects Cheminformatics Machine learning Virtual drug screening