https://www.euro-online.org/enog/inoc2007/Papers/mac-slots.html https://www.euro-online.org/enog/inoc2007/Papers/m https://www.euro-online.org/enog/inoc2007/Papers/mac-slots.html

Pretreatment Method of NIR spectra Reprocessing

sabki's picture
Forums: 

Dear all,

Which one is the best combinations and what is advantages every items of NIR spectra pretreatment method namely :

  1. 1st Derivative
  2. 2nd Derivative
  3. 3rd Derivative
  4. 4th Derivative
  5. Baseline Correction
  6. Detrend
  7. N-Point Smooth
  8. Multiplicative Scatter Correction
  9. Savitzky Golay

Thank You,

Regards,

Sabki

hlmark's picture

Sabki - if it were possible to answer your question, it would be unnecessary, That's because your question implies that there is one set of data pretreatments that was best for all applications under all conditions. Since that's not the case, there's no answer to your question. That's the short answer.

The long answer can be found in the many books and publications written over the years about NIR analysis. Your best approach is to learn more about NIR in general, and to find out what different people have found useful for samples similar to the ones you want to analyize. Thats takes work (in the lab and in the library) and strudy. There's no shortcuts to doing that.

\o/

/_\

 

gabiruth's picture

The question is so general that I don't think it is appropriate to start a long discussion.

However, I wish to make one basic comment

 

The most important thing to understand about a spectrum is that it is nothing more than a set of discrete values at certain discrete wavelengths. It is not a true mathematical function like f(x) = aX+bx2+ bX3  etc. where the derivatives are perfect continuous smooth fiunctions.

Derivative of a set of discrete set points is an approximation and in the process of doing it we create a lot of "noise". Thus, the more derivations we do we just build more and more noise. Therefore, in most cases, with few exceptions going into 2nd derivative is already risking that added noise will counter any possible benefit.

Thus, unless testing and finding that 2nd derivative results in better prediction capability, stay with firts derivative, and if you deal with clear liquids where there is no scattering, sometimes best is to stay with absorbance.

 

Gabi Levin

Brimrose Corp. of America

 

ianm's picture

Sabki, there are 6 essentials to successful application of NIRS. These are: excellent accuracy and precision in reference analysis; excellent spectral precision; excellent sample selection and sampling; and excellent presentation of the sample or material to the instrument. If there is a defect in any one of these the finest chemometrics in the world will not fully compensate. Excellent sample selection involves identification of all of the most important variables that can affect the spectrum, and assembling a set of samples that includes all of them, replicated at least 5 times, using different samples for each of the 5 replicates.

If you are confident that you have fulfilled all of these essentials, modern computers are very fast, so when you have assembled your spectra you can test any of these options in a few minutes. For practical purposes 3rd and 4th derivatives are not recommended, because as you increase the derivative you reduce the signal-to-noise ratio, which is not good. Both 1st and 2nd derivative achieve baseline correction, as well as end-point smoothing, and most derivative software uses the Savitsky/Golay procedure. It is a good idea to apply multiplicative scatter correction. It may, or may not improve your statistics, but it also a good idea to test no data pre-processing (i.e. original log 1/R spectral data) as well as derivative pre-treatment, sometimes it works just as well.

Remember that the objective of an application of NIRS is to arrive at a calibration reliable enough to use in industry, not to simply obtain attractive statistics, which you can obtain by deleting those pesky outliers! There is no easy way.

Good luck, and keep asking questions.

Phil.   
(sent on behalf of Phil Williams by Ian Michael)