Published February 15, 2023
FMCSA Declined to Adopt Item Response Theory (IRT) Model, but Proposed Changes to BASICs
After 12 years of using the Compliance, Safety, Accountability Safety Measurement System (SMS) to identify high-risk motor carriers for interventions, FMCSA is proposing significant changes to the current SMS methodology. After a full review and testing of the National Academy of Sciences’ proposed IRT statistical model developed under Congressional direction from the FAST Act in 2015, the FMCSA rejected the model stating that it “does not perform well for the Agency’s use in identifying motor carriers for safety interventions and does not improve overall safety.”
During the analysis and testing of the IRT model, FMCSA identified areas in the SMS methodology in need of improvement to better identify high-risk carriers. As a result, subject to public comment, the FMCSA is proposing the following changes to the SMS:
- Changes to the BASICs to realign and reorganize into “safety categories,” including new segmentation;
- Consolidated violations and groupings;
- Simplified violation severity weights;
- Proportionate percentiles to address safety event grouping issues;
- Adjusting intervention thresholds;
- Greater emphasis on recent violations; and
- An updated utilization factor
The most notable proposed changes would reorganize the BASICs into “safety categories” to better identify specific safety problems and to combine the 959 violations currently used in SMS, plus 14 new additional violations, into 116 violation groups. The proposal to simplify violation severity weights would change the current model, which weighs violations on a scale of 1 to 10, to a new approach that would assign a severity weight of either 1 or 2. Through its proportionate percentile approach, FMCSA is attempting to address the significant frustrations motor carriers have had as they move between safety event groups which often results in significant negative swings. Through safety category segmentation and adjusting the intervention thresholds, FMCSA is attempting to address inequities in the current SMS methodology based on the type of equipment carriers operate. The time weighting change would not score carriers in any safety category where a violation does not exist in the most recent 12 months. Lastly, the Agency is proposing to raise the utilization factor mileage cap from 200,000 to 250,000 miles.
While the FMCSA declined to adopt the IRT Model, the proposed changes to the SMS could have a significant impact on the way motor carriers are evaluated and how the Agency conducts interventions.
Contact Scopelitis Attorneys Tim Wiseman, Prasad Sharma, or Brigitte Collier with questions.
News from Scopelitis is intended as a report to our clients and friends on developments affecting the transportation industry. The published material does not constitute an exhaustive legal study and should not be regarded or relied upon as individual legal advice or opinion.
Published February 15, 2023
FMCSA Declined to Adopt Item Response Theory (IRT) Model, but Proposed Changes to BASICs
After 12 years of using the Compliance, Safety, Accountability Safety Measurement System (SMS) to identify high-risk motor carriers for interventions, FMCSA is proposing significant changes to the current SMS methodology. After a full review and testing of the National Academy of Sciences’ proposed IRT statistical model developed under Congressional direction from the FAST Act in 2015, the FMCSA rejected the model stating that it “does not perform well for the Agency’s use in identifying motor carriers for safety interventions and does not improve overall safety.”
During the analysis and testing of the IRT model, FMCSA identified areas in the SMS methodology in need of improvement to better identify high-risk carriers. As a result, subject to public comment, the FMCSA is proposing the following changes to the SMS:
- Changes to the BASICs to realign and reorganize into “safety categories,” including new segmentation;
- Consolidated violations and groupings;
- Simplified violation severity weights;
- Proportionate percentiles to address safety event grouping issues;
- Adjusting intervention thresholds;
- Greater emphasis on recent violations; and
- An updated utilization factor
The most notable proposed changes would reorganize the BASICs into “safety categories” to better identify specific safety problems and to combine the 959 violations currently used in SMS, plus 14 new additional violations, into 116 violation groups. The proposal to simplify violation severity weights would change the current model, which weighs violations on a scale of 1 to 10, to a new approach that would assign a severity weight of either 1 or 2. Through its proportionate percentile approach, FMCSA is attempting to address the significant frustrations motor carriers have had as they move between safety event groups which often results in significant negative swings. Through safety category segmentation and adjusting the intervention thresholds, FMCSA is attempting to address inequities in the current SMS methodology based on the type of equipment carriers operate. The time weighting change would not score carriers in any safety category where a violation does not exist in the most recent 12 months. Lastly, the Agency is proposing to raise the utilization factor mileage cap from 200,000 to 250,000 miles.
While the FMCSA declined to adopt the IRT Model, the proposed changes to the SMS could have a significant impact on the way motor carriers are evaluated and how the Agency conducts interventions.
Contact Scopelitis Attorneys Tim Wiseman, Prasad Sharma, or Brigitte Collier with questions.
News from Scopelitis is intended as a report to our clients and friends on developments affecting the transportation industry. The published material does not constitute an exhaustive legal study and should not be regarded or relied upon as individual legal advice or opinion.