Commission Delegated Regulation (EU) 2024/436 of 20 October 2023 supplementing Regulation (EU) 2022/2065 of the European Parliament and of the Council, by laying down rules on the performance of audits for very large online platforms and very large online search engines

SECTION I

General provisions

Article 1

Subject matter

This Regulation lays down rules on the performance of audits pursuant to Article 37 of Regulation (EU) 2022/2065, as regards:

(a) the procedural steps for ensuring that the auditing organisation to be selected fulfils the conditions laid down in Article 37(3) of Regulation (EU) 2022/2065;

(b) the procedural steps for cooperation and assistance by the audited provider in the performance of audits, including accessing relevant information with a view to obtaining audit evidence;

(c) the definition and selection of auditing methodologies;

(d) the templates for the audit report and the audit implementation report.

Article 2

Definitions

For the purpose of this Regulation, the following definitions shall apply:

(1) ‘auditing organisation’ means an individual organisation, a consortium or other combination of organisations, including any sub-contractors, that the audited provider has contracted to perform an independent audit in accordance with Article 37 of Regulation (EU) 2022/2065;

(2) ‘audited service’ means a very large online platform or a very large online search engine designated in accordance with Article 33 of Regulation (EU) 2022/2065;

(3) ‘audited provider’ means the provider of an audited service which is subject to independent audits pursuant to Article 37(1) of that Regulation;

(4) ‘audited obligation or commitment’ means an obligation or commitment referred to in Article 37(1) of Regulation (EU) 2022/2065 which forms the subject matter of the audit;

(5) ‘audit criteria’ means the criteria against which the auditing organisation assesses compliance with each audited obligation or commitment;

(6) ‘audit evidence’ means any information used by an auditing organisation to support the audit findings and conclusions and to issue an audit opinion, including data collected from documents, databases or IT systems, interviews or testing performed;

(7) ‘misstatement’ means an intentional or unintentional omission, misrepresentation or error in the declarations or data reported or provided by the audited provider to the auditing organisation, or in the testing environment made available by the audited provider to the auditing organisation;

(8) ‘audit risk’ means the risk that the auditing organisation issues an incorrect audit opinion or reaches an incorrect conclusion concerning the audited provider’s compliance with an audited obligation or commitment, considering detection risks, inherent risks and control risks with respect to that audited obligation or commitment;

(9) ‘detection risk’ means the risk that the auditing organisation does not detect a misstatement that is relevant for the assessment of the audited provider’s compliance with an audited obligation or commitment;

(10) ‘inherent risk’ means the risk of non-compliance intrinsically related to the nature, the design, the activity and the use of the audited service, as well as the context in which it is operated, and the risk of non-compliance related to the nature of the audited obligation or commitment;

(11) ‘control risk’ means the risk that a misstatement is not prevented, detected and corrected in a timely manner by means of the audited provider’s internal controls;

(12) ‘materiality threshold’ means the threshold beyond which deviations or misstatements by the audited provider, individually or aggregated, would reasonably affect the audit findings, conclusions and opinions;

(13) ‘reasonable level of assurance’ means a high but not absolute level of assurance, which allows the auditing organisation to assert in its audit opinion and audit conclusions whether the audited provider complies with the audited obligations or commitments, based on sufficient and appropriate evidence;

(14) ‘internal control’ means any measures, including processes and tests, that are designed, implemented and maintained by the audited provider, including its compliance officers and management body, to monitor and ensure the audited provider’s compliance with the audited obligation or commitment;

(15) ‘vetted researcher’ means a researcher vetted in accordance with Article 40(8) of Regulation (EU) 2022/2065;

(16) ‘audit procedure’ means any technique applied by the auditing organisation in the performance of the audit, including data collection, the choice and application of methodologies, such as tests and substantive analytical procedures, and any other action taken to collect and analyse information to collect audit evidence and formulate audit conclusions, not including the issuing of an audit opinion or of the audit report;

(17) ‘test’ means an audit methodology consisting in measurements, experiments or other checks, including checks of algorithmic systems, through which the auditing organisation assesses the audited provider’s compliance with the audited obligation or commitment;

(18) ‘substantive analytical procedure’ means an audit methodology used by the auditing organisation to assess information to infer audit risks or compliance with the audited obligation or commitment.

Article 3

Scope of the audit and reasonable level of assurance

SECTION II

Conditions for the performance of the audit

Article 4

Selection of the auditing organisation

Where the auditing organisation to be selected consists of more than one legal person or intends to have recourse to one or several sub-contractors, the audited provider shall check whether all those legal persons or subcontractors:

(a) individually fulfil the requirements laid down in Article 37(3), points (a) and (c), of Regulation (EU) 2022/2065;

(b) jointly fulfil the requirement laid down in Article 37(3), point (b), of Regulation (EU) 2022/2065.

Article 5

Cooperation and assistance between the audited provider and the auditing organisation

At a time agreed with the auditing organisation, and in any event prior to the performance of any audit procedure, the audited provider shall transmit to the selected auditing organisation at least the following information:

(a) a description of the internal controls put in place with respect to each audited obligation and commitment, including related indicators and all present and historical measurements, and benchmarks used by the audited provider to assert or monitor compliance with the audited obligations and commitments, as well as any supporting documentation;

(b) its preliminary analysis of inherent and control risks, where the audited provider has performed such an analysis, and any supporting documentation;

(c) information about any relevant decision-making structures, competences of departments of the provider, including the compliance function pursuant to Article 41 of Regulation (EU) 2022/2065, relevant IT systems, data sources, processing and storage, as well as explanations of relevant algorithmic systems and their interactions.

SECTION III

Performance of audits

Article 6

Audit report and audit implementation report

Article 7

Procedures for the preparations for the audit

The audited provider and the auditing organisation shall conclude a written agreement setting out:

(a) the exhaustive list of audited obligations and commitments;

(b) the responsibilities of the audit organisation, including, where applicable, detailed for each legal person constituting the auditing organisation, and the parties empowered to sign the audit report;

(c) the procedures and contact points made available by the audited provider for the auditing organisation to request access to data referred to in Article 5(2);

(d) the timeframe for the audit, including the start and end date of the audit procedures and the completion of the audit report;

(e) a procedure on how disputes between the audited provider and the auditing organisation arising from the performance of the audit shall be resolved.

Article 8

Audit opinion, audit conclusions and recommendations

The audit report shall include the audit conclusions that the auditing organisation has reached on the audited provider’s compliance with each of the audited obligations and commitments. The audit conclusions shall be either:

(a) ‘positive’, where the auditing organisation concludes with a reasonable level of assurance that the audited provider has complied with an audited obligation or commitment;

(b) ‘positive with comments’, where the auditing organisation concludes with a reasonable level of assurance that the audited provider has complied with an audited obligation or commitment, but: (i) the auditing organisation includes remarks on the benchmarks provided by the audited provider pursuant to Article 5(1), point (a); or (ii) the auditing organisation recommends improvements that do not have a substantive effect on its conclusion;

(c) ‘negative’, where the auditing organisation concludes with a reasonable level of assurance that the audited provider has not complied with an audited obligation or commitment.

Audit opinions pursuant to paragraphs 4 and 5 shall be either:

(a) ‘positive’ if the auditing organisation has reached a ‘positive’ audit conclusion for all of the audited obligations or commitments;

(b) ‘positive with comments’ if the auditing organisation has reached at least one audit conclusion that is ‘positive with comments’ for an audited obligation or commitment and has not reached a ‘negative’ audit conclusion for any of the audited obligations or commitments;

(c) ‘negative’ if the auditing organisation reached a ‘negative’ audit conclusion for at least one audited obligation or commitment.

SECTION IV

Audit methodologies

Article 9

Audit risks analysis

The audit risk analysis shall consider:

(a) inherent risks;

(b) control risks;

(c) detection risks.

The audit risk analysis shall be conducted taking into account:

(a) the nature of the audited service and the societal and economic context in which the audited service is operated, including probability and severity of exposure to crisis situations and unexpected events;

(b) the nature of the obligations and commitments;

(c) other appropriate information, including: (i) where applicable, information from previous audits to which the audited service was subjected; (ii) where applicable, information from reports issued by the European Board for Digital Services or guidance from the Commission, including guidelines issued pursuant to Article 35(2) and (3) of Regulation (EU) 2022/2065, and any other relevant guidance issued by the Commission with respect to the application of Regulation (EU) 2022/2065; (iii) where applicable, information from audit reports published pursuant to Article 42(4) of Regulation (EU) 2022/2065 by other providers of very large online platforms or of very large online search engines operating in similar conditions or providing similar services to the audited service.

Article 10

Appropriate audit methodologies

The audit report shall include a description of the audit methodologies designed by the auditing organisation prior to performing any audit procedures, including at least:

(a) the audit criteria, for assessing compliance with each audited obligation or commitment, defined on the basis of information pursuant to Article 5(1), point (a), and the materiality threshold tolerated and expressed in qualitative or quantitative terms, as appropriate;

(b) all tests and substantive analytical procedures and audit evidence that the auditing organisation intends to use to assess compliance for each audited obligation or commitment.

The audit report shall include a description of any changes to the methodologies used during the performance of the audit compared to the methodologies designed prior to performing audit procedures.

Reasonable doubts referred to in paragraph 3 shall be deemed to arise, in particular, in the presence of any of the following elements:

(a) professional judgment and scepticism in assessing information, including concerning internal controls of the audited provider, that leads the auditing organisation to formulate reasonable doubts;

(b) external indications pointing to audit risks, in particular reports from the European Board for Digital Services referred to in Article 35(2) of Regulation (EU) 2022/2065, guidance from the Commission including through guidelines referred to in Article 35(3) of that Regulation, and any other relevant guidance issued by the Commission with respect to the application of Regulation (EU) 2022/2065, and audit reports issued pursuant to codes of conduct or crisis protocols referred to in Articles 45, 46 and 48 of that Regulation;

(c) information related to events occurring during the performance of the audit, including crisis situations, that require additional actions from the audited provider to ensure compliance with certain audited obligations or commitments.

Audit procedures shall include at least:

(a) the performance of tests and substantive analytical procedures for the internal controls the audited provider has put in place for each of the audited obligations or commitments;

(b) the performance of substantive analytical procedures to assess compliance with each audited obligation and commitment, including as regards algorithmic systems;

(c) the performance of tests, including with respect to algorithmic systems, concerning the audited obligations and commitments in relation to which the auditing organisation has reasonable doubts, as referred to in paragraph 4, and concerning audited obligations and commitments where the auditing organisation deems necessary to perform tests in its choice of methodology pursuant to paragraph 1.

Article 11

Quality of audit evidence

The audit conclusions and audit opinions shall be based on audit evidence which fulfils both of the following requirements:

(a) it is relevant and sufficient to reduce audit risks identified in accordance with Article 9, and to enable the auditing organisation to provide audit conclusions and opinions in accordance with Article 8;

(b) it is reliable, according to the auditing organisation’s professional judgment and scepticism.

Article 12

Sampling methods

The sample size and methodology for sampling shall be selected in a way that ensures representativeness of the data or information and, as appropriate, in consideration of all of the following:

(a) the representativeness of the sample for the period referred to in Article 3(2) and (3);

(b) relevant changes to the audited service during that period;

(c) relevant changes to the context in which the audited service is provided during that period;

(d) relevant features of algorithmic systems, where applicable, including personalisation based on profiling or other criteria;

(e) other relevant characteristics or partitions of the data, information and evidence under consideration;

(f) the representation and appropriate analysis of concerns related to particular groups as appropriate, such as minors or vulnerable groups and minorities, in relation to the audited obligation or commitment.

Article 13

Specific methodologies for auditing compliance with Article 34 of Regulation (EU) 2022/2065 on risk assessment

The assessment of the audited provider’s compliance with Article 34 of Regulation (EU) 2022/2065 shall include, but not be limited to, an analysis of all of the following:

(a) whether the audited provider has diligently identified, analysed, and assessed the systemic risks in the Union referred to in Article 34(1), first subparagraph, of Regulation (EU) 2022/2065, including by assessing: (i) how the audited provider identified the risks that are linked to its service, taking into account regional and linguistic aspects of the use made of its service, including when specific to a Member State, and whether the risks are appropriately identified; (ii) how the audited provider analysed and assessed each risk, including how it considered the probability and severity of the risks, and whether the assessment was appropriate; (iii) how the audited provider identified, analysed and assessed the factors referred to in Article 34(2), first subparagraph, of Regulation (EU) 2022/2065, whether they were appropriately identified, and to what extent such factors influence the risks identified in paragraph 1 of that Article; (iv) what sources of information the audited provider used, how it collected the information, including whether and how it relied on scientific and technical insights; (v) whether and how the audited provider tested assumptions on risks with groups most impacted by the specific risks;

(b) whether the risk assessment was performed within the timeframes set out in Article 34(1), second subparagraph, of Regulation (EU) 2022/2065 and, where applicable, within the timeframes set for activities established as risk mitigation measures for the detection of systemic risks pursuant to Article 35(1), point (f) of that Regulation;

(c) how the audited provider identified functionalities that are likely to have a critical impact on the risks for which risk assessments shall be conducted prior to their deployment, pursuant to Article 34(1), second subparagraph, of Regulation (EU) 2022/2065, whether those functionalities were correctly identified, and whether the risk assessment was appropriately conducted;

(d) whether the audited provider correctly identified the supporting documentation that should be preserved with respect to the risk assessment and whether it has put in place the necessary means to ensure the preservation of that documentation for at least three years, pursuant to Article 34(3) of Regulation (EU) 2022/2065, and whether the documentation was preserved accordingly.

Without prejudice to any other analysis necessary for reaching a reasonable level of assurance, methodologies for auditing compliance with Article 34 of Regulation (EU) 2022/2065 shall include at least an assessment by the auditing organisation of the following elements:

(a) the internal controls that the audited provider has put in place to monitor the performance of risk assessments regarding each factor referred to in Article 34(2), first subparagraph, of Regulation (EU) 2022/2065; such assessment shall: (i) be based on substantive analytical procedures, for those internal controls; (ii) be based on tests of whether those internal controls are reliable and diligently conceived, executed and monitored; (iii) evaluate how the compliance officer or officers performed their tasks with respect to Article 41(3), points (b), (d), (e) and, where applicable, (f), of Regulation (EU) 2022/2065 and how the management body of the audited provider was involved in the decisions related to risk management pursuant to Article 41(6) and (7) of that Regulation;