Contact us

Live Chat with Tek representatives. Available 6:00 AM - 4:30 PM

Call

Call us at

Available 6:00 AM – 5:00 PM (PST) Business Days

Download

Download Manuals, Datasheets, Software and more:

DOWNLOAD TYPE
MODEL or KEYWORD

Feedback

What are 'counts' in the accuracy specification?

Question :

What are 'counts' in the accuracy specification?

Answer :

Product(s) :
Digital Multimeters: 175,175A,197,197A

Micro-ohm Meter: 580

CV/IV Meters: 595,590

Low-Current/High-Resistance Products: 6514, 6517, 6517A, Model 65

 


Question : What are ‘counts’ in the accuracy specification?


Answer: Unless stated otherwise, 1 count is equal to 1 unit of resolution of the selected range

The two terms in the specification are the zero and gain terms. Where % of reading is the gain term and the counts is the zero or offset term.

Here is an example of how to calculate accuracy when measuring a 0.5V signal on the 1V range when that range has a 10uV resolution.

The specification is stated as: 3% + 5 counts

This translates to 3% * (signal input) + 5 * (resolution of that range)

0.03*0.5 + 5*10uV = .015V + 50uV = 0.015050 = 15.05 mV

For the applied 0.5 volts, the reported measurement will be within specification as long as it is in the range of .48495 and .51505


©Copyright 2003, Keithley Instruments, Inc.


This FAQ Applies to:

No product series

Product:

FAQ ID 69831

View all FAQs »