How Far Beyond the Training Range Can ML Be Trusted?
Hi! One of the questions we get quite often is: "Can you trust an ML forecast when reservoir attributes fall outside what the model saw in training?" The short answer is yes — if you know how far outside the training range you actually are. Here is why.
We tested ML recovery factor prediction on 302 real layers.
Setup: We picked the highest-porosity layer and gradually shrank the training set — removing the most porous layer each step until only 50 remained. At each of 252 steps we retrained the model and predicted recovery factor for the same test layer.
What we found:
- Right at the edge of the training range, the model already overpredicts recovery factor by 13%.
- Around 28% beyond the training maximum, the forecast is almost perfect.
- Beyond 43%, the model flips to underprediction — up to 37% error.
Assuming a critical recovery factor error of 25%, the practical boundary looks like this:
- You can generally trust the forecast while the layer stays within roughly 35% above the maximum porosity seen in training. Below that, error stays under 25% most of the time.
- The risk zone is 35–50% beyond the training edge — that is where error repeatedly breaks 25%, in both directions.
- A small step outside the range (under 15%) does not automatically mean a bad forecast. The relationship is not linear — accuracy can dip, recover, and dip again as the training set changes.
So yes, you can trust ML prediction, we have wide training dataset, that covers the ranges of attributes
