ADC resolution vs accuracy

DAQ devices are sold by bit count, which sets the smallest step the converter can report but not how close a reading is to the true voltage. A 24-bit device can be wrong by millivolts, and a 16-bit device with a tight accuracy spec can beat a looser 24-bit one on absolute DC measurements. The ADC resolution calculator works out step size and noise-free resolution for any range and bit count.

Resolution: the size of one LSB

One least significant bit (LSB) is the full-scale span divided by the number of codes:

LSB = span / 2^N

On a ±10 V range the span is 20 V, so a 16-bit converter steps in 20 / 65536 = 305 µV.

Range counts as much as bit count. The same 16-bit converter that steps in 305 µV on ±10 V steps in 30.5 µV on ±1 V. Some devices also lose resolution in particular modes. The MCC USB-1208FS-Plus is 12-bit in differential mode but 11-bit in single-ended mode on its ±10 V range, a step of 9.77 mV. Plug your own range and bit count into the calculator to see the step in volts and in your engineering units.

Noise: effective and noise-free resolution

The converter and the amplifier ahead of it add noise, and once the noise is larger than one LSB the lowest bits are random.

Effective resolution, as LabJack uses the term, is the span divided by the RMS noise, expressed in bits: log2(span / noise_rms). Noise-free resolution uses peak-to-peak noise instead. For Gaussian noise, peak-to-peak is roughly 6.6 times the RMS value, so noise-free resolution comes out about 2.7 bits lower. LabJack publishes both for each T7 range and resolution setting. On ±10 V, effective resolution runs from 16.0 bits (316 µV) at resolution index 1 to 19.1 bits (37 µV) at index 8, with higher indexes taking longer per sample.

ENOB (effective number of bits) is the AC version, measured with a sine-wave test: ENOB = (SINAD - 1.76) / 6.02, where SINAD is the signal to noise and distortion ratio in dB. A converter with 74 dB SINAD has an ENOB of 12 bits, whatever its nominal resolution. Audio and dynamic signal hardware tends to quote ENOB or SNR, while general-purpose DAQ datasheets more often give noise in µV RMS or in counts.

The NI USB-6211 lists random noise on its ±10 V range as 229 µV RMS, about 0.75 LSB. Peak-to-peak that is about 1.5 mV, so its noise-free resolution is log2(20 V / (6.6 * 229 µV)), about 13.7 bits. Averaging 100 readings cuts random noise by a factor of 10, which NI assumes in its accuracy formula below. Sample rate and aliasing covers oversampling and averaging in more depth.

Accuracy: offset, gain, nonlinearity and drift

Accuracy is how far the reported value can be from the true input. A DAQ accuracy spec is built from several terms:

• Offset error: the reading with zero volts applied. It is the same across the range, so it is given in volts or in ppm of range.
• Gain error: a slope error that grows with the reading, given in % or ppm of reading.
• Integral nonlinearity (INL): the worst deviation from a straight line once offset and gain are removed.
• Temperature drift: offset and gain coefficients in ppm per °C, applied to the temperature change since calibration.
• Noise: random, and reducible by averaging.

The bit count appears in none of those terms. The analog front end, the voltage reference and the calibration set them.

The NI USB-6211 accuracy table

NI publishes an absolute accuracy formula and table for the USB-6211, a 16-bit, 250 kS/s multifunction device. From its specifications:

AbsoluteAccuracy = Reading * GainError + Range * OffsetError + NoiseUncertainty
GainError        = ResidualAIGainError + GainTempco * TempChangeFromLastInternalCal
                   + ReferenceTempco * TempChangeFromLastExternalCal
OffsetError      = ResidualAIOffsetError + OffsetTempco * TempChangeFromLastInternalCal
                   + INL_Error
NoiseUncertainty = RandomNoise * 3 / sqrt(100)

For the ±10 V range the table gives a residual gain error of 75 ppm of reading, gain tempco 7.3 ppm/°C, reference tempco 5 ppm/°C, residual offset error 20 ppm of range, offset tempco 34 ppm of range/°C, INL error 76 ppm of range and random noise of 229 µV RMS. NI’s stated conditions are 1 °C since the last internal (self) calibration, 10 °C since the last external calibration, 100 averaged points and a 3σ coverage factor. At a full-scale reading of 10 V:

GainError   = 75 + 7.3*1 + 5*10 = 132.3 ppm  -> 10 V * 132.3 ppm = 1323 uV
OffsetError = 20 + 34*1 + 76    = 130 ppm    -> 10 V * 130 ppm   = 1300 uV
Noise       = 229 uV * 3 / 10                                    =   68.7 uV
Total                                                            ~ 2692 uV

NI’s table lists 2,690 µV for the range, about 0.027% of full scale or roughly 9 LSB. The other ranges follow the same pattern:

The offset term is a fraction of the range, so a 1 V signal on the ±10 V range still carries the full 1,300 µV of offset. Its total is about 1,500 µV, 0.15% of the reading, against at most 310 µV on the ±1 V range.

Temperature change since calibration outweighs resolution. If the device has warmed 10 °C since its last self-calibration, the offset term grows to 436 ppm and the total to about 6.4 mV. Run self-calibration after warm-up to keep that term small.

The noise term assumes 100 averaged points. A single reading carries 3 * 229 = 687 µV of noise at 3σ instead of 68.7 µV.

A simpler table: MCC USB-1208FS-Plus

MCC states accuracy for the USB-1208FS-Plus as a percentage of reading plus a fixed offset. In differential mode on ±10 V that is 0.2% of reading plus 9.766 mV, a total of 29.766 mV at full scale, about 6 LSB of its 12-bit converter. On the ±1 V range the offset stays at 9.766 mV and the full-scale total is 11.766 mV, more than 1% of the range. The device suits signals measured in volts, not millivolts, and more resolution wouldn’t help because the offset spec is the limit.

Why a 24-bit converter can still be millivolts off

The NI-9239 is a 24-bit delta-sigma C Series module with a ±10 V nominal range (±10.52 V typical), so one LSB is about 1.25 µV. NI’s calibrated accuracy at 25 °C ±5 °C is typically ±0.03% of reading plus ±0.008% of range. At a 10 V reading that is 3.0 mV + 0.84 mV, about ±3.8 mV. Over its full −40 to 70 °C operating range the maximum is ±0.13% of reading plus ±0.06% of range, about ±19 mV at 10 V. Its input noise is 70 µV RMS.

The module resolves changes of a few microvolts while its absolute reading can be several millivolts off. For vibration, acoustics and other dynamic measurements that is the right trade, since resolution, noise and linearity count for more there than DC accuracy. For comparison, LabJack lists the T7’s absolute accuracy on ±10 V as ±0.01% of full scale, ±2,000 µV, with a 15 ppm/°C temperature coefficient.

Choosing a range that uses the converter well

• Use the smallest range that holds the largest signal you expect, including faults. Thermocouples belong on ±0.1 V or smaller. A 1–5 V transmitter signal fits a ±5 V range only if fault currents won’t push it past 5 V (see the 4–20 mA post).
• Convert the accuracy into engineering units. At about 40 µV/°C, the USB-6211’s 88 µV on ±0.2 V is about 2.2 °C for a type K thermocouple before cold junction error. The thermocouple guide has the full budget.
• Average when noise dominates and the signal is slow. Averaging reduces random noise and leaves offset and gain error untouched.
• Calibrate the system when the datasheet isn’t good enough. A two-point correction in software, offset from a shorted input and gain from a known reference, removes most of the offset and gain error at the temperature you calibrated at. DAQAtlas, which is in development, computes math channels on every sample, so the correction can run on live data.
• Compare accuracy tables, not bit counts. Fine resolution doesn’t move a reading closer to the true value.