Sure, but what I was saying was that what good is a new ranking method, when you only have at your disposal the same set of metrics as the method you are trying to replace? A new ranking would quite often mean adding new metrics. For example, when Lucene when from tf-idf to bm25 they added lots of new metrics to be able to cater for the new algorithm.