We've been away for some time. We tried to make some changes during that time, like moving the announcements, and news about the API, and open data to GitHub for public access. Progress has been minimal, and we'd like to take a moment to explain where things are, immediate changes, and near future ones coming up.
Why has there been little progress?
After taking over the Open Data initiative, and the API, there were priority and staffing changes that meant a significant loss of time assigned to the Open Data initiative. As the strategic priorities of the University and the technical group were defined and changed, we had to evaluate where we spent limited resources. While Open Data is an important asset, and supports initiatives by staff, students, and our community, there was no immediate need to make progress while v2 of the API was operational.
What will happen with the existing legacy v2 Open Data API?
It has remained in a legacy, critical support only state for most of 2019, and through 2020. At this stage it is on hardware, and software that is going out of support by the end of the year, and must be retired. As such, it will remain operational until the end of 2020, and be shutdown as soon as reasonably possible after that date. The v2 of the API has had tremendous success with getting data out to the community, and helping to develop some amazing projects, and solve data problems. We hope to build on that with more accurate, high quality data, going forward.
What is the status of the v3 Open Data API?
We released what we called a technical preview, and it has been live and in use by a limited amount of clients for over a year. We've now published a significant set of changes to the API endpoints, and the endpoint schemas. While we recognize this is a breaking change, based on the usage, and the changes needed to support getting the information into the API this was necessary. We'll be posting more details going forward, the biggest changes and the reasoning behind them are the following:
- On the academic data side (Quest); we removed any reference to effective dating of data. We'll be providing only the snapshot of data that is true "today". Having the flexibility to see the data as it was true in the past added complexity in exposing it, and had a significant educational component to make sure data was appropriately used by the clients and downstream systems.
- For academic data, and with other data as possible, we will be aiming for a range of near real time to sub-2 hour refreshes for the current, past, and next term. In all other cases data should be considered archived.
- There will be a significant decrease in initial endpoints available, as no authoritative data steward or system exists.
What is the status of "Crowd" data API?
We originally planned to have the data sources we could not find a data steward for, or the data was community curated offered through a "Crowd" API. This plan is indefinitely on hold at this time.
What changes will we be making before the end of the year?
We'll be updating the documentation across our sites (https://uwaterloo.ca/api/, https://uwaterloo.ca/open-data/, https://github.com/uWaterloo/OpenData), releasing more updates for v3 of the API, and turning off v2.
Where do I leave comments and feedback?
The easiest place to voice your questions, comments, and opinions is by filing an issue on our GitHub page (https://github.com/uWaterloo/OpenData/issues). If you wish for your communication to be less public, please open a request ticket through RT (https://rt.uwaterloo.ca/) for the IST-Opendata queue.