Skip to content

farbodab/Covid19CanadaArchive

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Archive of Canadian COVID-19 Data

THE DATA IN THIS ARCHIVE HAVE MOVED TO A NEW SERVER. To locate data, please use the file explorer: http://data.opencovid.ca/archive/index.html#archive/.

The root directory of the file server is: http://data.opencovid.ca/archive/. For example, the PHAC Epidemiology Update from November 4, 2020 may be downloaded at the following URL: http://data.opencovid.ca/archive/can/epidemiology-update-2/covid19-download_2020-11-04_23-38.csv.
From now on, this archive will be used only to store and collorate on archival scripts and metadata. Links to the data have been updated. Other options to explore and download the data will be available soon.

This repository provides automated, daily backups of COVID-19 data from Canadian governmental and non-governmental sources.

File name timestamps are given in ET (America/Toronto) in the following format: %Y-%m-%d_%H-%M. The script is run nightly around 23:00 ET.

All code in this repository is covered by the MIT License. Licenses and terms of use for each archived dataset are given below.

This repository is maintained by Jean-Paul R. Soucy on behalf of the COVID-19 Open Data Working Group.

Table of contents:

Contribution guide

Community members may contribute to the project in several ways. In the future, more ways of contributing will be added (e.g., adding metadata).

Add a new dataset

New datasets may be added in the following ways:

  • Create a pull request on GitHub adding the dataset to the appropriate location in the "active" section of data/datasets.json. See other entries for examples.
  • Create an issue on GitHub requesting the new dataset be added.
  • Email the maintainer requesting the new dataset be added.

If you have archived versions of the dataset you are adding (e.g., you previously downloaded the dataset daily), see "Contributing historical data" below.

Retire an inactive dataset

Some datasets continue to exist at a URL but are no longer updated. These datasets should be removed from the nightly update. This may be achieved in the following ways:

  • Create a pull request on GitHub moving the dataset's entry from the "active" section of data/datsets.json to the appropriate location in the "inactive" section. Also, change the dataset's "active" flag from "True" to "False". See other entries for examples.
  • Create an issue on GitHub requesting the dataset be retired.
  • Email the maintainer with the historical data.

Contribute historical data

Historical data (e.g., archived versions of a dataset newly added to the archival tool) may be contributed in the following ways:

  • Create an issue on GitHub regarding the historical data.
  • Email the maintainer regarding the historical data.

Recommended citation

COVID-19 Canada Open Data Working Group. Archive of Canadian COVID-19 Data. https://github.com/ccodwg/Covid19CanadaArchive. (Access date).

Running archiver.py

archiver.py can run in two modes:

  • python archiver.py prod: Download files and upload them to the server.
  • python archiver.py test: Don't upload files to the server, just test that they can be successfully downloaded.

The script relies on setting environmental variables to function properly. See archiver.py for more details.

Data sources/terms of use/supplementary material

The sources and terms of use for each included dataset are linked below. Supplementary material such as data dictionaries and codebooks are also included in the list below, if available. These files are included with the relevant datasets in a directory named supplementary.

Alberta

Edmonton

British Columbia

Canada

Manitoba

Winnipeg

New Brunswick

Newfoundland and Labrador

Northwest Territories

Nova Scotia

Nunavut

Ontario

Toronto

Ottawa

Quebec

When both French and English data files are available, French files should generally be considered definitive (and in many cases, these files have been captured in the archive for a longer duration). The English versions of files avaiable in both languages will always have their directories marked with "-en" at the end.

Montreal

Prince Edward Island

Saskatchewan

Yukon

Other: Non-governmental sources

Canada

Quebec

COVID-19 Canada Open Data Working Group

Data from the COVID-19 Canada Open Data Working Group is being added on an experimental basis. The full catalogue of historical data will be available in the future.

Data notes

On several occasions, the nightly archival script has failed to run. Depending on when the failure was identified, this may have resulted in a partial or total loss of archival data for that day. A list of these days is provided below:

  • Wednesday, October 21, 2020
  • Thursday, November 19, 2020

In the future, a package will be provided to more easily access the data provided in this archive and to document missing or incomplete data.

Acknowledgements

Many people are to thank for contributing archived data and code to this repository.

About

Archive of Canadian COVID-19 Data

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%