DKPS method cuts model-evaluation queries using cached responses

An arXiv preprint introduces a method that predicts a new model’s benchmark performance by reusing cached responses from previously evaluated models and applying a Data Kernel Perspective Space (DKPS) to model inter-model relationships. The paper frames this as response reuse plus a DKPS-based predictor that can estimate benchmark scores without generating a fresh answer for every query, addressing the practical cost of exhaustive evaluation in modern frameworks.

The DKPS mechanism quantifies relationships between black-box models to support interpolation of a target model’s behavior from existing outputs. The authors provide theoretical arguments that DKPS-based methods are query-efficient under certain conditions, and they report empirical results where the DKPS predictor achieves the same mean absolute error as baseline methods while using a substantially decreased query budget. They also add an offline procedure for selecting which queries to run: by maximizing goodness-of-fit on reference models, that selection outperforms random query sampling and improves prediction accuracy.

Operationally, the approach offers a way to shrink compute and time costs in evaluation pipelines by combining cached outputs with a targeted set of new queries, which could matter for teams that run frequent benchmarks or operate under API/query limits. The paper’s claims rest on theoretical conditions and on experiments summarized in the preprint; those conditions and experimental details are not specified in the brief preview, so further validation across benchmarks and model families is needed. Watch for follow-up work or released tooling that shows how robust DKPS predictors are in practice, and for any integration into benchmark suites and evaluation services that would make cached-response evaluation a standard option.