EMPLOYMENT AND TRAINING ADMINISTRATION
ADVISORY SYSTEM
U.S. Department of Labor
Washington, D. C. 20210

CLASSIFICATION

TAA

CORRESPONDENCE SYMBOL

ONR

ISSUE DATE

August 6, 2003

RESCISSIONS

None

EXPIRATION DATE

Continuing

ADVISORY

:

TRAINING AND EMPLOYMENT GUIDANCE LETTER NO. 3-03

 

TO

:

ALL STATE WORKFORCE LIASONS
ALL STATE WORKFORCE AGENCIES

 

FROM

:

EMILY STOVER DeROCCO
Assistant Secretary

 

SUBJECT

:

Data Validation Policy for Employment and Training Programs

 

  1. Purpose. To inform states and National Program grantees of the Employment and Training Administration’s data validation policy, and the plans to implement data validation requirements for the following programs: Workforce Investment Act Title IB, Labor Exchange, Trade Adjustment Assistance, Migrant and Seasonal Farmworkers, Native American Employment and Training, and Senior Community Service Employment Program. The data validation policy and supplemental guidance on implementing data validation will be conveyed to National Program grantees through the issuance of program bulletins.

  2. References. Training and Employment Notice (TEN) No. 14-02, "Data Validation Initiative;" TEN No. 8-02, "Implementation of Common Performance Measures for Job Training and Employment Programs;" and President’s Management Agenda ( http://www.whitehouse.gov/omb/budget/fy2002/mgmt.pdf).

  3. Background. The accuracy and reliability of program reports submitted by states and grantees using Federal funds are fundamental elements of good public administration, and are necessary tools for maintaining and demonstrating system integrity. The President’s Management Agenda to improve the management and performance of the Federal government has emphasized the importance of complete information for program monitoring and improving program results.

    States and grantees receiving funding under the Workforce Investment Act (WIA) Title IB, Labor Exchange, Trade Adjustment Act (TAA), and Senior Community Service Employment Program (SCSEP) are required to maintain and report accurate program and financial information [WIA Section 185 (29 USC 2935) and WIA regulations 20 CFR 667.300(e)(2); Wagner Peyser Act Section 10 (29 USC 49), Older Americans Act Section 503(f)(3) and (4) (42 USC 3056a (f)(3) and (4)), and TAA regulations 20 CFR 617.57]. Further, all states and grantees receiving funding from the Employment and Training Administration (ETA) are required to submit reports or participant records to the agency and attest to the accuracy of the data.

    Recent performance audits conducted by the Department of Labor’s Office of the Inspector General, however, found that the accuracy of reported performance outcomes cannot be assured due to insufficient local, state, and Federal oversight. To address this concern and meet the agency’s goal for accurate and reliable data, ETA committed to the development and implementation of a data validation process in order to ensure the accuracy of data collected and reported on program activities and outcomes.

    ETA has already begun implementing data validation requirements for the Unemployment Insurance program. ETA will separately issue requirements for reporting validation results for state programs under WIA Title IB, Labor Exchange, and TAA, and for the National Program grantees operating Migrant and Seasonal Farmworker (MSFW) programs, Native American Employment and Training programs, and SCSEP in fall 2003.

    States and grantees should be aware that there are legislative and policy changes on the horizon that will impact ETA’s performance accountability system. First, ETA plans to implement a set of common performance measures for job training and employment programs in CY 2004. Further information on the common measures can be found in TEN No. 8-02. Second, WIA was authorized for only five years, and the congressional process to pass reauthorizing legislation is underway. These legislative and policy changes will not significantly alter ETA’s data validation policy or the validation process. The validation tools developed by ETA, discussed later in this guidance, will be adapted to reflect changes to data collection and WIA reauthorization.

  4. Employment and Training Administration Data Validation Policy.

    1. General. In order to ensure the accuracy and reliability of reported information, states and grantees are required to validate the data submitted to ETA. Failure to demonstrate the validity of reported data will be deemed to be a failure to report, and subject to corrective action or sanction, as appropriate.

    2. Scope. There are two basic sources of reporting error. First, if the data collected are incorrect or data entry errors occur, then the outcome information will not be accurate. Second, even if the data collected are correct, if the state’s or grantee’s reporting system does not meet Federal standards, it could calculate the performance outcomes incorrectly. Therefore, the ETA data validation requirement consists of two parts:

      1. Report validation evaluates the validity of aggregate reports submitted to ETA by checking the accuracy of the reporting software used to calculate the reports. Report validation is accomplished by processing an entire file of participant records into validation counts and comparing the validation counts to those reported by the state or grantee.

      2. Data element validation assesses the accuracy of participant data records. Data element validation is performed by reviewing samples of participant records against source documentation to ensure compliance with Federal definitions.

    3. Frequency. Data validation will be required annually as follows:

      1. Report validation must be completed prior to the submission of required reports to ETA (exceptions to this for the initial year of validation are noted in Sections 6 and 7).

      2. Data element validation must be completed within 120 days after required annual reports or participant records are submitted to ETA. Exact deadlines for the completion of data validation will vary somewhat by program.

    4. Accuracy Standards. For the data validation initiative to be effective and to allow for continuous improvement, ETA will establish acceptable levels for the accuracy of reports and data elements. These accuracy standards will be established in phases. The initial validation year will focus on detecting and resolving any issues with state and grantee data and reporting systems. Error rates collected in the second year will be analyzed, and based on this information, standards for accuracy will be established prior to the third year of validation.

      Once accuracy standards are established, states and grantees will be held accountable for meeting those standards and will be required to address any issues concerning data accuracy. States and grantees that fail to meet accuracy standards will receive technical assistance from ETA and will develop and implement a corrective action plan. Data that do not meet accuracy standards will not be acceptable for measuring performance, and may keep the state or grantee from being eligible for incentives that are awarded based on performance data. Significant or unresolved deviation from accuracy standards may be deemed a failure to report.

  5. Description of ETA Data Validation Process. The ETA data validation process is intended to accomplish the following goals:

    ETA has developed a set of validation tools discussed in Section 8 – instructional handbooks, software, and user guides – that states and grantees can use to validate data. The ETA software can also be used to generate the aggregate information required in reports submitted to ETA. States and grantees that use the software provided by ETA to generate this aggregate information will not be required to perform report validation.

    States and grantees may use an alternative methodology and tools as long as the methodology meets criteria for sampling and confidence intervals. States and grantees that do not use the validation tools provided by ETA will be required to document that the alternative methodology is statistically valid.

    At present, two states have participated in a pilot and ETA is beginning to work with additional states that are interested in being early implementers of the data validation process. Early implementing states have been testing the validation methodology using PY 2001 data. All states and grantees are encouraged to begin testing validation as well.

  6. Implementation for State Programs. Reporting instructions for data validation for state programs will be issued separately. The timeframes for implementation are as follows:

    In subsequent years, states will be required to complete report validation for WIA Title IB and Labor Exchange prior to the submission of the reports listed above to ETA. However, in this initial year of validation, report validation will be completed in the same timeframe as data element validation – by April 1, 2004, which is approximately 120 days after the submission of the reports to ETA.

  7. Implementation for National Grant Programs. Reporting instructions for data validation for national grant programs will be issued separately. The timeframes for implementation are as follows:

  8. Data Validation Tools. ETA has developed instructional handbooks, standardized software, and user guides that states and grantees can use to perform data validation.

    ETA’s validation handbooks, software applications, and user guides for each program can be found on ETA’s Web site at http://www.doleta.gov/usworkforce/performance/. The validation tools for the WIA Title IB, Labor Exchange, and TAA programs are currently available; tools for the national grant programs will be posted as finalized.

  9. Training and Technical Assistance. ETA is providing data validation training for states in regional sessions during the summer of 2003. Training for National Program grantees will be provided separately for each program. Training for grantees of the MSFW and Native American Employment and Training programs will be held during winter 2003/2004, and training will be provided for SCSEP grantees during spring 2004.

    States and grantees are encouraged to obtain technical assistance on validation procedures and the use of the validation tools by contacting ETA’s data validation contractor. To avoid unnecessary delays and problems in the implementation of data validation, the contractor is available to facilitate the validation process and to address any difficulties states and grantees encounter. E-mail and phone contact information is available on the ETA Web site listed above, in the Help menu of each software application, and in the user guides.

  10. Monitoring. To ensure that states and grantees are able to successfully implement data validation procedures and standards, ETA will monitor the validation effort on a regular schedule. The record retention requirements that will apply to records of all validation activities, including files, worksheets, reports, and source documentation, will be articulated in the reporting requirements to be issued separately.

  11. Action Required. States and grantees are expected to:

  12. Inquiries. Questions regarding the data validation initiative should be directed to the appropriate regional office or national grant program office.