An open-source API to scrape, process, and serve college course and class data 📒
(Also, a fresh take on OwlAPI =D)
- 🔍 Scrapes terms, departments, courses, and classes
- 🧩 Campus-agnostic architecture allows easy extensibility
- 📅 Currently serves 10+ years of just FHDA data
- 🏫 Scrapes a total of 4 colleges (adding more is just a matter of time)
Coming Soon:
- 📙 Scrape course catalogs to get more course info, such as descriptions
- 🔗 Link classes to professors for advanced analytics
- 🌐 GraphQL API (proof-of-concept finished)
Data Explorer => opencourse.dev
URL is subject to change.. but for now, the frontend and the API are hosted on the same domain.
Currently, the following data for the following campuses exists:
| id | name |
|---|---|
fh |
Foothill College |
da |
De Anza College |
wv |
West Valley College |
mc |
Mission College |
All endpoints (except for /:campus) support the following query parameters:
| name | format | default |
|---|---|---|
year |
a valid year | 2020 |
quarter |
fall, winter, spring, or summer |
fall |
Example: /da/depts?year=2019&quarter=fall
Example: /fh
Get campus metadata information.
Sample Response
{
"id": "fh",
"terms": [
{
"year": 2020,
"term": "summer",
"code": "202111"
},
{
"year": 2020,
"term": "spring",
"code": "202041"
},
{
"year": 2020,
"term": "winter",
"code": "202031"
},
{
"year": 2019,
"term": "fall",
"code": "202021"
},
{
"year": 2020,
"term": "fall",
"code": "202121"
}
]
}Example: /fh/courses
Get all courses at a campus for the selected term and year.
Sample Response
[
{
"dept": "ACTG",
"course": "1A",
"title": "Financial Accounting I"
},
{
"dept": "ACTG",
"course": "1B",
"title": "Financial Accounting II"
},
{
"dept": "MATH",
"course": "1A",
"title": "Calculus"
}
]Example: /fh/classes
Get all classes at a campus for the selected term and year.
Sample Response
[
{
"CRN": 20238,
"raw_course": "ACTG F001A01W",
"dept": "ACTG",
"course": "1A",
"section": "01W",
"title": "Financial Accounting I",
"units": 5,
"start": "10/19/2020",
"end": "12/11/2020",
"seats": 2,
"wait_seats": 15,
"status": "open",
"times": [
{
"days": "TBA",
"start_time": "TBA",
"end_time": "TBA",
"instructor": [
"Joe L Mayer (P)"
],
"location": "FC ONLINE"
}
]
},
{ "...": "..." }
]Get selected classes in a batch request.
{
resources: Resource[]
}Combinations: CRN, dept, or dept + course
{
CRN: number,
dept: string,
course: string
}{
resources: [
{
status: 'success' | 'error',
error: string,
data: Class | Class[] | null
}
]
}Sample Body & Response
Body:
{
resources: [
{ dept: 'CS', course: '1A' },
{ dept: 'CS', course: '2B' },
{ CRN: 20211 },
{} // Invalid
]
}Response:
{
"resources": [
{
"status": "success",
"data": [
{
"CRN": 30239,
"raw_course": "C S F001A01Z",
"dept": "CS",
"course": "1A",
"section": "01Z",
"title": "Object-Oriented Programming Methodologies in Java",
"units": 4.5,
"start": "01/04/2021",
"end": "03/26/2021",
"times": ["..."],
"status": "open",
"seats": 40,
"wait_seats": 10,
"wait_cap": 10
},
{"...": "..."}
]
},
{
"status": "success",
"data": [
{
"CRN": 30239,
"raw_course": "C S F001A01Z",
"dept": "CS",
"course": "1A",
"section": "01Z",
"title": "Object-Oriented Programming Methodologies in Java",
"units": 4.5,
"start": "01/04/2021",
"end": "03/26/2021",
"times": ["..."],
"status": "open",
"seats": 40,
"wait_seats": 10,
"wait_cap": 10
},
{"...": "..."}
]
},
{
"status": "success",
"data": {
"CRN": 30324,
"raw_course": "ACTG F001C03W",
"dept": "ACTG",
"course": "1C",
"section": "03W",
"title": "Managerial Accounting",
"units": 5,
"start": "01/04/2021",
"end": "03/26/2021",
"times": ["..."],
"status": "open",
"seats": 40,
"wait_seats": 15,
"wait_cap": 15
}
},
{
"status": "error",
"error": "At least \"CRN\" or \"dept\" have to be specified."
}
]
}Example: /fh/classes/20238
Get a class by its CRN (in a given term and year).
Sample Response
{
"CRN": 20238,
"raw_course": "ACTG F001A01W",
"dept": "ACTG",
"course": "1A",
"section": "01W",
"title": "Financial Accounting I",
"units": 5,
"start": "10/19/2020",
"end": "12/11/2020",
"seats": 2,
"wait_seats": 15,
"status": "open",
"times": [
{
"days": "TBA",
"start_time": "TBA",
"end_time": "TBA",
"instructor": [
"Joe L Mayer (P)"
],
"location": "FC ONLINE"
}
]
}Example: /fh/depts
Get all departments at a campus for the selected term and year.
Sample Response
[
{
"id": "GIST",
"name": "Geospatial Tech & Data Sci"
},
{
"id": "GLST",
"name": "Global Studies"
},
{
"id": "GID",
"name": "Graphic and Interact Desig"
}
]Example: /fh/depts/GLST
Get a department at a campus for the selected term and year.
Sample Response
{
"id": "GLST",
"name": "Global Studies"
}Example: /fh/depts/MATH/classes
Get all classes in a department for the selected term and year.
Sample Response
[
{
"CRN": 20086,
"raw_course": "MATH F001A01V",
"dept": "MATH",
"course": "1A",
"section": "01V",
"title": "Calculus",
"units": 5,
"start": "09/21/2020",
"end": "12/11/2020",
"seats": 0,
"wait_seats": 7,
"status": "waitlist",
"times": [
{
"days": "MW",
"start_time": "07:30 AM",
"end_time": "09:45 AM",
"instructor": [
"Diana Monica Uilecan (P)"
],
"location": "FH ONLINE"
}
]
},
{ "...": "..." }
]Example: /fh/depts/MATH/courses
Get all courses in a department for the selected term and year.
Sample Response
[
{
"dept": "MATH",
"course": "1A",
"title": "Calculus",
"classes": [
20086,
20087,
20088,
20257,
20761,
20762,
20305,
21310
]
},
{
"dept": "MATH",
"course": "1B",
"title": "Calculus",
"classes": [
20672,
20089,
20769,
20773
]
}
]Example: /fh/depts/MATH/courses/1A
Get a course in a department for the selected term and year.
Sample Response
{
"dept": "MATH",
"course": "1A",
"title": "Calculus",
"classes": [
20086,
20087,
20088,
20257,
20761,
20762,
20305,
21310
]
}Example: /fh/depts/MATH/courses/1A/classes
Get all classes for a course in the selected term and year.
Sample Response
[
{
"CRN": 20086,
"raw_course": "MATH F001A01V",
"dept": "MATH",
"course": "1A",
"section": "01V",
"title": "Calculus",
"units": 5,
"start": "09/21/2020",
"end": "12/11/2020",
"seats": 0,
"wait_seats": 7,
"status": "waitlist",
"times": [
{
"days": "MW",
"start_time": "07:30 AM",
"end_time": "09:45 AM",
"instructor": [
"Diana Monica Uilecan (P)"
],
"location": "FH ONLINE"
}
]
},
{ "...": "..." }
]Install requests, and use the API as follows:
import requests
API_URL = 'https://opencourse.dev'
req = requests.get(f'{API_URL}/fh/courses')
if req.ok:
courses = req.json()
for course in courses:
print(course['dept'], course['course'], course['title'])View Output
ACTG 1A Financial Accounting I
ACTG 1B Financial Accounting II
ACTG 1C Managerial Accounting
ACTG 51A Intermediate Accounting I
ACTG 52 Advanced Accounting
ACTG 53 Financial Statement Analysis
ACTG 54 Accounting Information Systems
ACTG 58 Auditing
ACTG 59 Fraud Examination
ACTG 60 Accounting for Small Business
ACTG 64A Computerized Accounting Practice Using Quickbooks
ACTG 64B Computerized Accounting Practice Using Excel
ACTG 65 Payroll & Business Tax Accounting
ACTG 66 Cost Accounting
ACTG 67 Tax Accounting
...
For running the API server, install python 3.8, pip, and pipenv.
Note: the following is subject to change (especially the scraper modules)
git clone https://github.com/OpenCourseAPI/OpenCourseAPI.git
cd OpenCourseAPI
pipenv install # install all python dependencies
python -m campus.fhda.fhda_scrape # scrape Foothill / De Anza College
python -m campus.wvm.wvm_scrape # scrape West Valley / Mission College
python server.py # start the serverTo run the frontend, install Node.js (preferably v12+) and yarn. Afterwards, run the following:
cd frontend
yarn install # install NPM packages
yarn start # start dev serverTo build for production, use:
yarn run buildThe generated static files are in frontend/build. To run a static server to serve the built files, use:
yarn run start:staticTo run tests, use the following:
yarn test # run all tests
yarn test:e2e # only run end-to-end testsAll contributions are welcome! A contribution guide is TBA.. but you can start by opening an issue and looking at the development guide above. Thanks!