dolseal title banner
Introduction
What to Expect
TPS Overview
   Computed Measures
   Systems Reviews
   Sampling
   Methods Surveys
Annual Report 		           Sampling
Welcome to the overview of TPS Sampling. Sampling is one components of TPS Program Review, the other component, Systems Review, is covered in the preceding module HERE.  This training section provides explanation of TPS sampling, the types of sampling, and steps involved. It also offers links to more details on sampling procedures for each of the major tax functions.


What Is the Purpose of Acceptance Sampling?
Acceptance Sampling confirms that each tax function’s system controls are producing accurate work/results.


What Types of Samples Will Be Reviewed?
Within the six major tax functions, thirteen categories of samples will be identified and reviewed :

  • Status Determinations
  • New Employers
  • Successor Employers
  • Inactivations/Terminations
  • Cashiering
  • Report Delinquency
  • Collections
  • Field Audit
  • Account Maintenance
  • Contribution Report Processing
  • Debts/Billings, Contributory
  • Debts/Billings, Reimbusing
  • Credits/Refunds
  • Benefit Charging
  • Tax Rates


What Steps Are Involved?
TPS Acceptance Sampling consists of four steps:

  • Building a universe of transactions
  • Extracting a sample of transactions
  • Evaluating the quality or accuracy of these transactions
  • Concluding Acceptance Sample reviews


Step 1.   Universe Building
In universe building, transactions that meet the criteria of the tax functions under review are identified and gathered. While most States have automated universe building programs in place, it is the reviewers responsibility to work closely with Data Processing staff to:

  • Ensure the universe building programs meet the universe criteria and that the data is valid.
  • Confirm automated universe building programs are producing a report that provides a total count of the universe records on the day the universe building is completed.
  • Identify simple verification sources for each universe count. For example: the total universe count for Status Determinations - New/Reopened Employers should match the count of New Status Determinations on the ETA 581 report (cell #14) for the calendar year.

HINT: Adjustments in universe building programs may be necessary if samples are extracted that do not meet the TPS sampling criteria or if changes are made to work processes within the State.

Step 2.  Sample Extraction
Since the purpose of Acceptance Sampling is to provide an assurance of quality, not an exact level of accuracy, small sample sizes can be used. Typically, 60 cases will be drawn from each tax function, once each year. (The Cashiering function is the exception in which estimation sampling has a target goal of 500 cases.)

Exactly how many cases will be sampled? It depends on the size of the universe.

The TPS Reviewer should work closely with data processing staff to ensure random sample selection. There are three numbers required for a proper sample:

1.  Total records in the universe - this count is provided by the automated universe building program or manually counted by the reviewer while gathering the universe.

2.  The number of cases to sample - depends on the size of the universe.

3.  A random number - a table of random numbers for use in TPS Acceptance Sampling is supplied by Regional Office prior to the beginning of each calendar year.

Depending on the State, these numbers may be totally automated or the reviewer may have to provide the numbers. Regardless of the method used, review the data collected:


INSPECT WHAT YOU EXPECT!

Do not over-sample!

HINT: Each sample should contain only the exact number of cases specified by ET Handbook 407. Over-sampling will produce inaccurate measurements.

Step 3. Sample Evaluation
Assemble appropriate information and source documents on each sample case as outlined in the Acceptance Sample instructions contained in ET Handbook 407.


 

HINT: The TPS reviewer must ensure that records are retained in all tax functions to such a degree as to allow all Acceptance Sampling questions to be answered.
Plan ahead - be sure records are retained!

It is essential to work with appropriate staff to ensure that all case documents or automated data files are available for review. For example: if the States system does not retain a copy of the billing statement issued to employers (imaged, microfilmed, or screen facsimile), the TPS reviewer should work with Data Processing staff to retain a copy of the print file. A copy of the billing statement needed for the Debits/Billings review can then be reproduced.

Evaluate the samples - use the Acceptance Sampling questions specifically designed for each tax function to evaluate each case. Some Acceptance Sampling questions are evaluative while others are non-evaluative. Acceptance Sample instructions list the evaluative questions and denote the non-evaluative questions with an asterisk (*).

HINT: It is important to evaluate each case in a consistent manner when applying TPS requirements. Reviewers should not be influenced by the wide variety of circumstances presented in sample cases.

HINT : Many Acceptance Sample questions ask if actions were taken according to State policy or guidelines. If a case does not meet established State criteria, the sample should consistently fail the TPS review. The reviewer may, however, conclude that a modification to State policy would be appropriate (based on effective Computed Measures results or other indicators) and include a recommendation within the TPS Annual Report segment.

If two or less cases out of sixty fail, then the sample passes.

How are Additional Findings handled? Discovery of findings that do not impact the pass/fail status of the review cases, but are worth noting, may be included as Additional Findings in each tax function. Additional Findings of a material nature may also be noted within the Annual Report.

When can a case be replaced? Click here for details.

Additional sampling - Acceptance Sampling design offers three options:

    1.  Accept the finding of the first Acceptance Sampling
    2.  Conduct a second Acceptance Sampling
    3.  Conduct an Expanded Sampling


If two or more cases within a tax function have missing case records or managers are not convinced the Acceptance Sampling findings of a specific function are correct, a second sample may be selected. Note: Only one additional sample may be selected and it replaces the prior sample for reporting purposes.

Expanded Sampling is used to estimate exact error rates, as opposed to Acceptance Sampling which merely tests whether the error rate in a function is below an acceptable level. Expanded sampling may be appropriate when management finds Acceptance Sampling results unconvincing or wishes to investigate the true level of errors or problems within a specific function.

Click here to link with ET Handbook 407 - “Other Sampling” specifications


Step 4. Concluding the Review
There are several steps in concluding the Acceptance Sampling review:

  • Record Findings - Within each tax function, results of the Acceptance Sample questions for each sample case should be recorded on sample coding sheets.  Reasons for each case failure should be recorded on sample explanation sheets.  Examples of these forms are provided at the end of each chapter of Acceptance Sample instructions within ET Handbook 407.
  • Data Entry - Acceptance Sampling results for the TPS review year are to be data entered into the DOL Sun System by March 31st of the following calendar year.
Click here to link to ET Handbook 407 - Appendix F - Data Entry User’s Guide.
  • Program Review Findings Chart - Acceptance Sample results should next be recorded in the Program Review Findings Chart.  This chart links sampling results with findings from Systems Reviews.  The chart should be included in the Annual Report.

Chart- Program Review Findings Chart - ET Handbook 407 - Chapter 2 - Figure II-4
  • Identify/Explain Any Inconsistencies - Acceptance Sampling results are not meant to stand alone.  Samples are a means of confirming the performance of the internal controls assessed in the Systems Reviews. Inconsistencies between the Acceptance Sampling results and Systems Reviews should be noted and explained within the Annual Report narrative.
  • Incorporate Findings From Computed Measures - Computed Measures results should also be considered in evaluating Acceptance Sampling results within the Annual Report narrative. Computed Measures reveal the timeliness of some functions and reflect the “big picture” effectiveness of other functions. For example: Acceptance Sampling results for New Employer Status Determinations may indicate a reasonable assurance of quality in the accuracy of the status decision and the posting of status information, but Computed Measures indicate that determinations are not being made in a timely manner.

    Collection samples may fail the Acceptance Sampling since the State does not file liens timely according to State policy.  But Computed Measures indicate an effective turnover ratio due to a high level of receivables liquidated as the result of other Collections methods.

  • Annual Report - The hard copy Annual Report is due to the Regional Office by May 31st. of the calendar year following the TPS Review year.  However, keying of review data into the SUN system and signing off on the data should be completed by March 31st.
Click here to link to ET Handbook 407 - Appendix D - Example of Annual Report.

Conclusion
Now that an overview of Acceptance Sampling has been explained, click on any of the following TPS tax functions for sampling details. To proceed to the final section - preparing the TPS Annual Report, click HERE.