Python in public-sector data analysis: reproducibility, auditability and personal data

Python is distributed under the Python Software Foundation License Version 2. This licence grants a non-exclusive, free, worldwide right to use, for reproducing, examining, testing, preparing derivative works from, and distributing Python. The condition is that the licence agreement and the copyright notice are preserved. Python's documentation states that all Python versions are open source.

The main libraries that accompany Python in data analysis are also distributed under free licences. Pandas, which enables work with tabular data, and GeoPandas, which adds geographic data support to pandas, are both licensed under BSD-3-Clause. Jupyter, which allows the explanatory text of analysis steps to be kept together with the results in a single document, is published under a licence the project describes as a modified BSD licence. Pandas has been among the projects supported by the non-profit organisation NumFOCUS since 2015, and GeoPandas since 2023; Jupyter is a non-profit project developed openly by community consensus.

The practical consequence of this licensing structure for institutions is that the analysis environment does not generate a cost tied to the number of users. An analysis prepared in one unit can be passed to another unit or another institution without requiring a licence-purchasing process, and opened and run there with the same tools.

Reproducibility means that an analysis carried out with the same data and the same method gives the same result every time. When the steps of the analysis are kept as a written script or notebook document, rather than as manual copying, filtering and formula operations, the document itself shows which data was used, which records were excluded and on what grounds, and which calculation was performed. This matters in the public sector for two reasons.

The first is audit. When the figures behind a planning decision or a report are questioned, the analysis can be rerun from the beginning to show whether the same result is reached. The second is continuity. When the staff member who prepared the analysis changes role, the work does not remain dependent on that person's memory; the same report can be reproduced the next period with current data, following the same steps.

To ensure reproducibility, the dated version of the data used, as well as the versions of Python and the libraries used, must also be recorded alongside the analysis document. Because the same steps can give a different result when library versions change, version information must be kept as an inseparable part of the analysis.

According to the Python Developer's Guide, a Python version receives bug and security fixes after release. After two years (18 months for versions before 3.13), only security fixes are accepted, and support ends five years after release. According to the schedule in the guide, as of 23 October 2026 the situation is as follows:

• Python 3.15: released on 9 October 2026, currently in the bugfix phase; support is planned to end in October 2031.
• Python 3.14: in the bugfix phase; support is planned to end in October 2030.
• Python 3.13, 3.12 and 3.11: receive security fixes only; support is planned to end in October 2029, October 2028 and October 2027 respectively.
• Python 3.10: support ended on 1 October 2026.

The version of Python used in an institution's analysis environments should, like other software components, be tracked in the inventory and updated before its support period expires. For analyses intended to be used over a long period, starting with a version still in the bugfix phase helps defer the need to upgrade.

A significant part of public-sector data contains personal data. Law No. 6698 on the Protection of Personal Data defines anonymisation as rendering personal data impossible to associate with an identified or identifiable natural person under any circumstances, even when matched with other data. Under Article 7 of the Law, personal data whose grounds for processing have ceased must be deleted, destroyed or anonymised. Article 28, first paragraph, subparagraph (b), excludes from the scope of the Law the processing of personal data for purposes such as research, planning and statistics through official statistics and anonymisation.

The Personal Data Protection Authority's Guide on the Deletion, Destruction or Anonymisation of Personal Data, dated April 2025, explains anonymisation methods with examples. The guide covers methods such as removing variables and records, regional suppression, generalisation and sampling, as well as statistical methods that strengthen anonymity, such as k-anonymity, l-diversity and t-closeness.

Reproducible analysis also provides convenience to the institution on this point. When the anonymisation steps are written into the analysis document, it is possible to audit which fields were removed, which values were generalised, and which thresholds were used. The authority under which personal data was transferred into the analysis environment, and how it will be stored after the analysis, must comply with the institution's personal data retention and destruction policy.

The output of an analysis prepared with open tools and properly anonymised is suitable for sharing with other units within the institution or with the public as open data. Providing the analysis steps alongside the result table when sharing guides users and researchers who wish to see how the data was produced. To this end, it is advisable for the institution to archive its analysis documents, the tool versions used and the anonymisation steps in a consistent format, to set a common practice across units, and to ensure that raw data containing personal data is kept separate from the shared version.

Establishing this practice within an institution is helped by setting the following minimum rules:

• Every analysis underlying a decision or a report is archived with its steps as a written script or notebook document.
• The source and date of the data used in the analysis, together with the Python and library versions, are kept together with the analysis document.
• In analyses involving personal data, the anonymisation steps are shown explicitly in the document and reviewed by a second person before sharing.
• The Python versions used in analysis environments are tracked in the software inventory and updated before their support period expires.