Format the values

For data from different sources and formats to be interoperable, it has to be adapted to the platform’s standard. Data Commons uses schemas — a common vocabulary for structuring and defining data — mainly to describe time series and statistical data.

Here’s an example of statistical data in the target shape:

entity	date	variable	value	unit
São Luís	2022	Adultos_atividade_fisica_regular	41,2	%
São Luís	2022	Adultos_atividade_fisica_regular_feminino	38,5	%
São Luís	2022	Adultos_atividade_fisica_regular_masculino	44,1	%
São Paulo	2022	Adultos_atividade_fisica_regular	46,8	%
São Paulo	2022	Adultos_atividade_fisica_regular_feminino	43,7	%
Rio de Janeiro	2022	Adultos_atividade_fisica_regular	39,9	%

Two things to notice:

1. Each distinct entity–variable pair is on its own row.
2. Only four columns are required: entity, date, variable, and value.

Every observation of a statistical variable is therefore situated in time and space.

Required columns

entity

Entities represent real-world objects the values are attached to. Each entity has a unique identifier — its DCID — and usually corresponds to some kind of place, though it can also represent things like a population or a school.

For Brazil, entities are already defined for city, state, and country:

• Cities and states use Wikidata identifiers — for example wikidataId/Q175.
• Countries use DCIDs of the form country/XXX, where XXX is the country’s ISO 3166-1 alpha-3 code.

Occasionally you may need other kinds of geographic region. See the knowledge graph concepts for how places relate to one another.

Each CSV file must contain only one type of entity. If you have data at both the municipality and the country level, split it into separate files.

Caution

If you truly need an entity type that doesn’t exist yet, it must be defined in the global schema. This is rare and should be avoided whenever possible.

date

The date of the event, in YYYY, YYYY-MM, or YYYY-MM-DD format.

variable

Variables are the statistical variables — they describe the phenomenon being measured. Like entities, each variable needs a DCID. Before defining a new one, check the Statistical Variable Explorer for an existing variable with the same properties — if one exists, reuse its name.

Extra columns for additional properties are not allowed in the standard CSV format. Instead, the variables themselves must encode those properties. For example, occupancy of hospital beds in public vs. private hospitals by age group becomes distinct variables:

Ocupacao_leitos_hospitais_publicos_adulto
Ocupacao_leitos_hospitais_privado_adulto
Ocupacao_leitos_hospitais_publicos_idoso
Ocupacao_leitos_hospitais_privados_idoso

value

Each value is a single observation of a variable for a given entity and date. Values must be numeric and contain no special characters. Zeros are accepted and recorded normally; for empty cells or non-numeric data, leave the field blank (or use Nan, Na, or N/A).

Optional columns

Column	Meaning
unit	Unit of measure for the value, giving context for interpretation. Use it when the unit isn’t implied by the variable’s definition.
observationPeriod	Period or frequency of the observations, in ISO-8601 (P[0-9][Y|M|D|h|m|s]). Annual is P1Y, monthly P1M, daily P1D.
measurementMethod	How the observations were produced (census, estimate, sample survey). An arbitrary string, or the DCID of a method already defined in Data Commons.
scalingFactor	Integer denominator for ratios or percentages. For values expressed as a percentage, the denominator is 100.

Column names are flexible

In the CSV, the order and names of columns don’t matter — what matters is that the column mapping is correctly defined in the config.json. However, the DCIDs of entities and variables must exist in Data Commons; otherwise you’ll need to define them in an MCF file.

The final CSV should look similar to this:

municipio,ano,sv,valor,unidade
dcid:wikidataId/Q42800,2022,dcid:br/Prevalence_PhysicalActivity_Person_Adult,41.2,%
dcid:wikidataId/Q42800,2022,dcid:br/Prevalence_PhysicalActivity_Person_Adult_Female,38.5,%
dcid:wikidataId/Q42800,2022,dcid:br/Prevalence_PhysicalActivity_Person_Adult_Male,44.1,%
dcid:wikidataId/Q649,2022,dcid:br/Prevalence_PhysicalActivity_Person_Adult,46.8,%
dcid:wikidataId/Q649,2022,dcid:br/Prevalence_PhysicalActivity_Person_Adult_Female,43.7,%
dcid:wikidataId/Q649,2022,dcid:br/Prevalence_PhysicalActivity_Person_Adult_Male,50.2,%
dcid:wikidataId/Q174,2022,dcid:br/Prevalence_PhysicalActivity_Person_Adult,39.9,%
dcid:wikidataId/Q174,2022,dcid:br/Prevalence_PhysicalActivity_Person_Adult_Female,36.4,%
dcid:wikidataId/Q174,2022,dcid:br/Prevalence_PhysicalActivity_Person_Adult_Male,43.6,%