This repository aims to show Austria's use of text classification model, for the classification of ISCO codes, based on a transformer model trained from scratch. This work was carried out within Cluster 2 of Work Package 10 "Text-to-Code" (WP10) of the AIMLL4OS project.
You can find more information about WP10 on the CROS website, its GitHub Repository and its dedicated GitHub Pages.
The report provides an introduction to transformer-based models for text classification in official statistics used at Statistics Austria. It explains the underlying transformer architecture, discusses practical considerations for training and evaluation, and outlines best practices for applying these models to automated coding tasks such as assigning NACE, ISCO, or ISCED classifications.
The exercises/ folder contains practical examples for training and evaluating transformer-based text classification models using publicly available data from Statistics Austria's classification data bank. The code scripts in the exercises/R contain implementations for data pre- and postprocessing, model building and training.
The file exercises/run_transformer_model.qmd guides users through the complete workflow, including data preprocessing, tokenization, model training, inference, and evaluation, providing reproducible examples that complement the concepts introduced in the accompanying report.
The runnable toy example can be run on an SSPCloud instance via this link
The report to our work can be found here.
This repository follows the AIML4OS template provided by the Work Package 6.