Introduction
RateProf-Scrape is a python class for scraping reviews about professors from a certain university from ratemyprofessors.com. This includes both aggregate information as well as detailed review scores (along the 4 different axis the website provides).Does this in the three steps: (a) Getting list of all professors whose last name starts with a specific letter; (b) Getting the aggregate statistics for each professor; (c) Getting individual reviews for each professor.
If interrupted, it will resume from where it stopped without having to download the previous files.
Detailed documentation of the code is available here.
Please contact the author (at the above address) for questions or to report bugs.
Download
Please read the license under which this code is distributed before using it.Download: [GZ] [ZIP].
There is also a Windows executable for installation (untested).
These packages contain the following files:
1. README.txt : The readme file for the project.
2. LICENSE.txt : License under which software is released.
3. scrapeRateProfs.py : The python module.
4. setup.py : The setup file.
5. DOCUMENTATION.html : The file containing detailed documentation of the code.
Compiling and Installation
RateProf-Scrape can works in Windows, Linux and Mac environment. It does require Python version 2.7 or newer in order to run properly.If you want to install this module in your directory for third-party Python modules then run (within the directory containing the code):
python setup.py install
To run the code, run:
python scrapeRateProfs.py [-h] -sid SchoolID [-delay DELAY] -o OUTPUT -path PATH
For more details see the readme.
Usage
Data Formats
Input: The function takes in input via the command line as mentioned above. The key input is the school id on ratemyprofessors.com.For example, Cornell University (found at www.ratemyprofessors.com/SelectTeacher.jsp?sid=298) has the sid of "298".
Output: (a) An aggregate TSV (Tab-seperated) file containing 1 professor's average ratings per line. The file will be sorted alphabetically as per last name. Details of the format are given in the README. (b) A TSV file containing all the reviews, one per line. This is grouped as per the faculty.
FAQ
Q) How do I find the ID of a school/university?A) Search for the university on ratemyprofessors.com and the look at the url. The "&sid=..." will give the required ID.