Modelagem e engenharia de dados para análise longitudinal de estabelecimentos brasileiros: uma plataforma de suporte à políticas públicas

Abstract

Mapping the Creative Economy in Brazil faces structural obstacles due to the inoperability of public data provided by the Federal Revenue Service. Although rich in information, the National Registry of Legal Entities (CNPJ) files are provided in massive and static formats, frequently containing encoding inconsistencies that make direct analysis unfeasible. Furthermore, the rigidity of the National Classification of Economic Activities (CNAE) does not reflect the transversality of creative sectors, and the lack of temporal traceability in raw data prevents evolutionary studies. This work aimed to develop, implement, and validate a scalable Web Platform for the ingestion, personalized classification, and temporal analysis of Brazilian establishments. The methodology adopted an applied research approach with incremental development, using a three-tier architecture composed of a Python/Flask Backend, React/Next.js Frontend, and PostgreSQL database. As main results, an ETL (Extract, Transform, Load) pipeline was implemented capable of sanitizing corrupted characters at the source, ensuring database integrity. Additionally, the system incorporates automatic scheduling mechanisms to ensure the continuous update of the historical series. The platform introduced a temporal data model, enabling the reconstruction of past scenarios and cohort analysis. The taxonomy management functionality provided methodological flexibility, allowing researchers to create their own groupings of economic activities. Technical validation confirmed the system's efficiency in processing millions of records and executing complex analytical queries in real-time, consolidating the platform as an effective tool for democratizing data access and strategically monitoring the Creative Economy