The aim of PharmBench is to provide a community benchmark dataset for the evaluation of molecular alignment and pharmacophore elucidation approaches, and also a webservice to help users evaluate their own methods.
The dataset consists of 81 targets, containing 960 ligands in total. The dataset is based on known pharmaceutically relevant co-crystallised protein ligand complexes, which were filtered to leave high-resolution structures containing drug-like small molecule ligands, for which electron density is also available. For each target the structures have been aligned using the receptor backbone atoms, and then the ligands extracted and prepared by fixing their atom types and adding hydrogen atoms.
The datasets therefore contain target ligands, aligned in the biologically relevant frame of reference, to act as a 'gold standard' reference set. The two-dimensional structures of these ligands are also provided.
An ideal molecular alignment or pharmacophore elucidation method should be able to read the two-dimensional input structures, and output three-dimensional alignments of the ligands that are equivalent to the gold standard alignments.
The citation for this work is: Cross, S.; Ortuso, F.; Baroni, M.; Costa, G.; Distinto, S.; Moraca, F.; Alcaro, S.; Cruciani, G. GRID-based 3D pharmacophores II: PharmBench, a benchmark dataset for evaluating pharmacophore elucidation methods. 2012, J. Chem. Inf. Model. 2012, 52(10):2599-2608
To evaluate alignment performance, a web service is available to score alignment models against the gold standard using the same metrics reported in the original paper:
- To measure how well the bioactive conformation of each ligand has been reproduced (rmsd)
- To measure the overall alignment quality by aligning the model to the gold standard using an iterative approach; the percentage of ligands that are correct (<2.0 A) is reported (50% is the objective measure reported in the paper)
- To measure the pharmacophoric similarity of the model compared to the gold standard in terms of their FLAPpharm Pharmacophoric Interaction Field similarities.