Measurable Skills Gains (MSG) Procedure (Updated 3/17/2023)

Overview:

The purpose of the Measurable Skill Gains (MSG) indicator is to track and measure important progressions and achievements made by WIOA clients (enrolled in training or education), through pathways and towards various goals of their individual programs.  A gain must be recorded for a client each year they are enrolled in training or education and progress satisfactorily in one or more areas of their employment or training program.  (For additional information on Measurable Skills Gains see US DOL TGEL 10-16 Change 2, pages 17-24:

https://www.dol.gov/sites/dolgov/files/ETA/advisories/TEGL/2022/TEGL%2010-16%20Change%202/TEGL%2010-16%20Change%202.pdf).

These procedures outline:

1. Who is in the measure
2. Types of Skills Gains
3. How to record Skills Gains in Career Connect

UPDATED! Who is in the measure?

1)  In-school Youth - ALL in-school youth are in the measure every program year until they exit WIOA.

2)  Out-of-School Youth who, during a Program Year (PY), enroll in a training program or secondary or post-secondary education leading to a diploma, high school equivalency, or certification. The customer must be enrolled in one of the following services:

• • 401-PARTNER PROGRAM REQUIRED: Registered Apprenticeship Program (RAP)
  • 406-Intergrated Education and Training (IET)/ICAPS
  • 408-Alternative Secondary School / Dropout Recovery Services
  • 416-Career Pathways Training (non-ITA)
  • 430-Tutoring / Study Skills Training Instruction / Dropout Prevention
  • 445-Other Non-Occupational Skills Training
  • 453-Occupational Classroom Training (ITA)
  • 400- Prerequisite Training Courses*
  • 403-Bridge Training (WIOA-funded, non-ITA)*
  • 432-Skills Upgrading/Retraining*
  • 458-Non-Registered Apprenticeship*

*These services were inactivated for Youth on 3/10/2023. However, any Youth who participated in one of these services prior to 3/10/2023 will be in MSG measure for every program year that the service is open.

3)  Adult and Dislocated Workers who, during a Program Year (PY) enroll in one of the following training services:

• • 300-Occupational Classroom Training (ITA)
  • 301-On-the-Job Training (OJT)-Private Sector
  • 302-On-the-Job Training (OJT)-Public Sector
  • 303-Career Pathways Training (non-ITA)
  • 305-Prerequisite Training Courses
  • 307-Bridge Training (WIOA-funded, non-ITA)
  • 310-Entrepreneurial Skills Training
  • 311-Customized Training
  • 315-Registered Apprenticeship Program (RAP)-ITA-funded
  • 324-Other Non-Occupational Skills Training
  • 326-Skills Upgrading & Retraining

Additionally, participants must meet specific requirements for certain skills gains types, as outlined below.

1.  

Time Frames for Inclusion in the MSG Indicator

• Skills Gains are based on the Program Year (July 1-June 30) and NOT on the Exit date. This means that Skills Gain is a live, in-program measure.

• Adult, DW and OSY customers are in the measure each year they participate in WIOA and are enrolled in training or education (i.e. have an open training service/education service as defined above).

• If the training/education service dates span multiple program years, then the participant is in the measure each year.

• EXAMPLES:
  • If Susie started Occupational Classroom Training in May of 2018 (PY17) and completed training in Oct 2018 (PY18), she would be included in the measure in both PY17 and PY18.
  • If William participates in training from January-May 2018 (PY17), but continues to receive job placement services through August 2018 (PY18), he is only included in the measure in PY17.

• Agencies must close completed training/education services in a timely manner to ensure that customers are removed from the measure upon training completion.

• Participants who are enrolled in training/education are included in the measure regardless of how long they have participated in the Program Year. For example, if a participant is enrolled in June, they have until June 30^th to achieve a gain.

• In accordance with DOL guidance, “Programs should not delay enrollment or services to participants until a new program year even if programs believe there is insufficient time for the participant to make any type of measurable skill gain by the end of that program year.”

Types of Measurable Skills Gains

Skills Gains are measured by five Skill Types, and may be documented as follows:

1)  Post-Secondary Transcript/Report Card:

• Applies to participants enrolled in post-secondary education
• Full-time students must complete a minimum of 12 hours per semester during the WIOA program year (July 1-June 30)
• Part-time students must complete a total of at least 12 credit hours over the course of two completed consecutive semesters during the WIOA program year
• Allowable documentation: Transcript or Report Card that shows a participant is meeting the State unit’s academic standards. The documentation must include whether the participant is enrolled full- or part-time. It must not show that the participant dropped out or was removed on academic or conduct grounds.

2)  Secondary Transcript/Report Card:

• Applies to participants without a high school diploma or equivalency at program entry
• Allowable Documentation: Participant’s secondary school transcript or report card for one semester showing that the participant demonstrating satisfactory achievement in all classes. It must not show that the participant dropped out or was removed on academic or conduct grounds. The semester must be within the Program Year.

3)  Training Milestone:

• Participant made satisfactory or better progress towards established milestones, such as completion of OJT (Adult/DW only) or completion of one year of an apprenticeship program or similar milestones. 
• Allowable Documentation:  Satisfactory or better progress report from an employer or training provider that is providing training (e.g., training reports showing completed milestones, increases in pay resulting from new skills, or increased performance.)

NEW! Although OJT and Work Based Learning (WBL) services do not put Youth in the MSG performance measure, a training milestone earned through an OJT or WBL may be counted as an MSG for ISY or for an OSY who is also enrolled in one of the training services listed above.  The gain must be outlined as an objective in the ISS prior to documenting the MSG.

For example, if an ISY graduates from high school in June 2023 (PY 22), but participates in a summer internship in July and August 2023 (PY 23), the Youth will still be in the MSG in PY 23.  A training milestone from the summer internship may be used to meet the PY 2023 MSG performance measure, as long as the milestone was outlined as an ISS objective in advance.  

4)  Skills Progression:

• Participant successfully passes an exam that is required for an occupation or progress in attaining technical or occupational skills as evidenced by trade-related benchmarks, such as knowledge based exams and/or a diploma. (e.g., employer knowledge based exam, occupational competency assessment, test necessary to obtain a credential.)
• Allowable Documentation: Copy of results of exam, test or assessment or copy of diploma or certificate. 

5)  Educational Functioning Level (EFL): An individual can achieve an EFL gain for MSG in one of two ways:

a. Test Scores: A Participant who is below post-secondary grade level at program entry and achieves a gain of at least one EFL on an acceptable Adult Basic Education (ABE) or English as a Second Language (ESL) post-test.

• Allowable Documentation: Pre- and post-ABE or ESL test. NOTE: EFL gains are calculated based on test scores entered on the WIOA “Educational Functioning Level for Measurable Skills Gains” bar in Career Connect.

b.  UPDATED! EFL Gain by Entry into Post-Secondary Education:  The EFL gain for entry into post-secondary education applies to Youth Exiters who:

• • were in the MSG due to enrollment in secondary education (high school / alternative school) or other non-occupational skills training, AND
  • whose school status at exit is “Not attending school; H.S. Graduate”, AND
  • who enter Post-Secondary education or training after exit, but during the same program year as the end date of the service that put them in the MSG (for out-of-school youth) or the same program year as the exit (for in-school-youth).

Although multiple gains may be entered for a client in a program year (July 1-June 30), only one gain will count towards performance per year.

Recording Measurable Skill Gains in Career Connect

1)  Recording non-EFL Skills Gains: All measurable skills gains other than EFL gains, are recorded in Staff Profile > Case Management Profile > Programs > WIOA Measurable Skills Gains bar. 

Step 1:

Click on the Measurable Skills Gains bar to expand it and then click on “Create Measurable Skills Gain”.

Step 2:

In the “General Information” section, select or confirm the LWDB and Office.

Step 3:

In the Skill Attainment Information, select the applicable Skill Type* based on the descriptions and requirements outlined above.

*NOTE: DO NOT select the “Credits Attained for EFL or Completed Secondary Ed and Enrolled in Post-Secondary Ed” Skill Type.  It does not transmit to IWDS and does not count for WIOA Performance. For participants who earn this skill gain, use the instructions in Part B of the “Entering Educational Functional Level (EFL)” section above.

Step 4:

Add the date the Skill Gain was attained in the “Date Skill Attained” box.

Step 5:

Select the “Type of Achievement” (see figure above).  Note that the Type of Achievement selections will vary depending on which Skill Type was selected in Step 2 above. The table below shows which Type of Achievements go with which Skill Types.

Step 6:

Click on “Verify” and select “Other Applicable Documentation, (specify).  Type in the documentation used to verify the skills gain in the box.

Step 7:

Add and save a case note providing details about when and how the skills gain was attained, including which WIOA service(s) led to the gain.

2)  Entering Educational Functioning Level (EFL) Gains: 

a. To record EFL Gains based on Adult Basic Education (ABE) and English as a Second Language (ESL) test scores. To record EFL gains, add pre-, progress- and post-test scores in the WIOA Educational Functioning Level for Measurable Skills Gain screen in accordance with the Literacy & Numeracy procedures posted at: https://workforceboard.zendesk.com/hc/en-us/articles/115006352528-Literacy-Numeracy-Procedures-Revised-9-28-2017- . See Appendices A and B of this procedure for lists of ABE and ESL tests scores and their corresponding EFL levels. 

b. To record EFL Gains for exiters who received training/education services and enter Post-Secondary after exit:

Step 1:

In the WIOA Closure screen select “Not attending school, HS Graduate” in the “School Status at Exit” field. Be sure that you have proof of school status at exit in the customer file. (NOTE: this field is not required for Adult/DW exiters, but must be entered to receive credit for the MSG.)

Step 2:

When creating a quarterly follow-up record, set the “Placement in Quarter” field to “Post-secondary Education”, indicate the date of placement in post-secondary, and indicate the documentation used to verify post-secondary enrollment. 

NOTE: The post-secondary placement must occur during the same program year as the qualifying training/education service (for OSY) or the same program year as exit (for ISY) to count as a MSG.

NOTE: Career Connect uses a different methodology to calculate “EFL Entry into Post-secondary Education” than IWDS does. The data entry instructions above are to ensure that the MSG is counted correctly in IWDS because IWDS data is used for federal reporting and WIOA Performance. IWDS and Career Connect MSG reports may not match up exactly due to the differences.

UPDATE! Reports

There are two different reports that agencies can use to see which of their participants have earned a Measurable Skills Gain.

1.  Measurable Skill Gains Indicators – Predictive Report

The MSG Indicators Predictive report will show all your participants who are in the measure for a given program year and which ones have attained an MSG and which ones have not. 

Report Path:  Detailed Reports > Case Management Reports > Predictive > WIOA > Measurable Skills Gains Indicators

Filters:

• Include Columns:  select as desired to include participant demographic and eligibility information.  Leave unchecked if you do not want this information.
• Program:  Title I – WIOA
• Customer Group:  Select as desired; leave as “None Selected” to include all customer groups
• Region/LWDB:  Chicago Cook Workforce Partnership
• Office Location:  Select your office(s)
• State:  Change to “None Selected” if you have participants from out-of-state
• Date Range Type:  Select Rolling 4 Quarters

Date Range:  Select Fourth Quarter and the Program Year.  For example, to see all participants in the measure for PY 2022 (July 1, 2022-June 30, 2023), select Fourth Quarter and 2022. 

Report Results:

The report results include all participants who are in the MSG measure for the selected program year, whether or not they have earned the MSG.  The report includes columns indicating the start and end date of the Education or Training that put the participant in the MSG measure, and a column for each type of service activity that put the participant in the measure. 

In the example below, the Participant started their training in April of 2021 and completed it in September 2022 and the training was an OJT or Registered Apprenticeship.

The report also includes columns for each type of MSG, including an EFL Gain.  If a gain was recorded in a category, the MSG date will display in the column.  In this case the participant attained a Training Milestone MSG on 9/16/2022.

The other columns that can help identify which participants still need an MSG are the Exclude, Numerator, and Denominator columns:

• Exclude: A “Y” in the Exclude column means that the participant has been excluded from performance because of a neutral exit (such as Health/Medical, Incarceration, etc).
• Denominator:  A “Y” in the denominator means that the participant is included in the MSG measure.  Unless the participant was excluded based on an exit reason, all participants included in the report are in the Denominator.
• Numerator:  A “Y” in the Numerator means that the participant has successfully attained an MSG in the program year, and thus represents a positive performance outcome.  If the numerator column is “N” then the participant is a negative outcome on the MSG measure unless an MSG is recorded by the end of the program year.  These are the participants you should review to ensure you are meeting your performance goals.

In the example below, our OJT participant is in both the Numerator and Denominator because they successfully attained (and staff recorded) an MSG for the OJT.

The predictive report includes additional columns of information, but the examples above show the columns to use to ensure your participants are earning skills gains and that those gains are being recording in the system.

2.  Measurable Skill Gains  - Case Management Report

The Measurable Skill Gains case management report returns all participants who have earned an MSG in the current program year.  Unlike the Predictive report described above, the case management report does not include participants who are in the MSG measure, but do not have a skills gain recorded.

Report Path: Go to Detailed Reports > Case Management Reports > Case Load and select “Measurable Skill Gains”

Filters:

• Program: Title I-Workforce Development (WIOA)
• Customer Group: Select as desired; keep as “None Selected” to return all customer groups
• Region/LWDB:  Chicago Cook Workforce Partnership
• Office Location: Select your office(s)
• Program Year: Select the desired program year. Remember that the program year is based on the Program Year start date of July 1. For example, for the period July 1, 2017 to June 30, 2018, select Program Year 2017.

Report Results:

After clicking “Run Report” Career Connect will return a summary of total number of participants who have made each Skills Gain Type, including the Educational Functional Level (EFL) gain in the selected program year (July 1 – June 30).

To see which participants have made the gains, click on the blue number in the “Total Skill Gain” column.

Clicking the number will link to a “child” report showing the participant-level details.

For general resources on how to run reports in Career Connect, including a guide on commonly used reports, see: https://workforceboard.zendesk.com/hc/en-us/articles/6503046470541.