https://www.euro-online.org/enog/inoc2007/Papers/mac-slots.html https://www.euro-online.org/enog/inoc2007/Papers/m https://www.euro-online.org/enog/inoc2007/Papers/mac-slots.html

Independent Validation Test

sabki's picture
Forums: 

Hello All,

I had develop the NIR model using OPUS software with good results as below : R2val : 96%, R2cal : 97% RPD : 5 RMSECV : low.

I used 100 samples in the model and scanned 4 times each samples ( I got 400 sprectras). The samples are very homogenous with the same particles size. The lab reference data also cover the range that we want tested model (independent validation test).

When I do independent validation test (new samples) the result still not satisfied (R2<30%) so that can not used model for screening material (Ranking) :

1. What is the solution to investigation of the model?

2. Test set 20% in the model of 100 samples also good why independent validation test still not good?

Thank you so much for your assist

Kind Regards,

Sabki

 

ptillmann's picture

Dear Sabki,

two possible answers to your question:
- Either the two sample sets are not coming from the same population and differ so much that the independent valdiation set cannot be predicted from the calibration samples.
- or the calibration models noise - or some interfering substance  - in the calibration samples which are not present to the same extend in the validation samples.

In the first case you might plot in 2D or 3D to view both sample sets and check whether they fall apart or not.
In the second case you might reduce the no of factors and see whether the reduced models predict well.

Both scenarios are frequently encountered.

 

Keep calibrating

Peter

sabki's picture

Hi Peter,

Thanks for your suggest. I have done it with PCA to see some of samples in independent test still not cover in the model.

Regards,

Sabki