2 __Description__: A django application for sharing and uploading class notes.
4 __Copyright__: FinalsClub, a 501c3 non-profit organization
6 __License__: GPLv3 except where otherwise noted
8 __Contact__: info@karmanotes.org
10 v3.0 of the karmanotes.org website from the FinalsClub Foundation
14 KarmaNotes is an online database of college lecture notes. KarmaNotes empowers college students to participate in the free exchange of knowledge.
20 Before doing anything, you'll need the code. Grab it from github.
22 Clone the project from the central repo using your github account:
24 git clone git@github.com:FinalsClub/karmaworld.git
26 If you aren't using a system setup for github, then grab the project with
29 git clone https://github.com/FinalsClub/karmaworld.git
31 Generally speaking, this will create a subdirectory called `karmaworld` under
32 the directory where the `git` command was run. This git repository directory
33 will be referred to herein as `{project_root}`.
35 There might be some confusion as the git repository's directory will likely be
36 called `karmaworld` (this is `{project_root}`), but there is also a `karmaworld`
37 directory underneath that (`{project_root}/karmaworld`) alongside files like
38 `fabfile.py` (`{project_root}/fabfile.py`) and `README.md`
39 (`{project_root}/README.md`).
41 ## External Service Dependencies
43 Notice: This software makes use of external third party services which require
44 accounts to access the service APIs. Without these third parties available,
45 this software may require considerable overhaul. These services have
46 API keys, credentials, and other information that you must provide to KarmaNotes
47 as environment variables. The best way to persist these environment variables is
48 by using a `.env` file. Copy `.env.example` to `.env` and populate the fields as
52 This project has chosen to use [Heroku](www.heroku.com) to host the Django and
53 celery software. While not a hard requirement, the more up-to-date parts of this
54 documentation will operate assuming Heroku is in use.
56 See README.heroku for more information.
59 The instructions for creating an [S3](http://aws.amazon.com/s3/) bucket may be
60 [found on Amazon.](http://docs.aws.amazon.com/AmazonS3/latest/gsg/CreatingABucket.html)
62 Two, separate buckets will be needed in production: one for static file hosting
63 and one as a communication bus with Filepicker.
65 This software uses S3 to store files which are sent to or received
66 from Filepicker. Filepicker will need to know the S3 bucket name, access key,
69 Filepicker users can only make use of an S3 bucket with a paid account. For
70 development purposes, no Filepicker S3 bucket is needed. Skip all references to
71 the Filepicker S3 bucket in the development case.
73 The software will not need to know the S3 credentials for the Filepicker
74 bucket, because the software will upload files to the Filepicker S3 bucket
75 through Filepicker's API and it will link to or download files from the
76 Filepicker S3 bucket through Filepicker's URLs. This will be covered in the
77 Filepicker section below.
79 This software uses S3 for hosting static files. The software will need to
80 update static files on the S3 bucket. As such, the software will need the
81 S3 bucket name, access key, and secret key via the environment variables. This
82 is described in subsections below.
84 To support static hosting, `DEFAULT_FILE_STORAGE` should be set to
85 `'storages.backends.s3boto.S3BotoStorage'`, unless there is a compelling reason
88 There are three ways to setup access to the S3 buckets depending upon speed
89 and security. The more secure, the slower it will be to setup.
91 #### insecure S3 access
92 For quick and dirty insecure S3 access, create a single group and a single user
93 with full access to all buckets. Full access to all buckets is insecure!
96 [Amazon IAM group](http://docs.aws.amazon.com/IAM/latest/UserGuide/Using_CreatingAndListingGroups.html)
97 with full access to the S3 bucket. Select the "Amazon S3 Full Accesss" Policy
101 [Amazon IAM user](http://docs.aws.amazon.com/IAM/latest/UserGuide/Using_SettingUpUser.html).
102 Copy the credentials into the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`
103 environment variables. Be sure to write down the access information, as it
104 will only be shown once.
106 #### secure S3 access
107 For secure S3 access, two users will be needed. One with access to the
108 Filepicker bucket and one with access to the static hosting bucket.
110 Note: this might need to be modified to prevent creation and deletion of
114 [Amazon IAM group](http://docs.aws.amazon.com/IAM/latest/UserGuide/Using_CreatingAndListingGroups.html)
115 with full access to the S3 bucket. The quick way is to select the
116 "Amazon S3 Full Accesss" Policy Template and replace `"Resource": "*"` with
117 `"Resource": "arn:aws:s3:::<static_bucket_name>"`.
120 [Amazon IAM user](http://docs.aws.amazon.com/IAM/latest/UserGuide/Using_SettingUpUser.html).
121 Copy the credentials into the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`
122 environment variables. Be sure to write down the access information, as it
123 will only be shown once.
125 Ensure the created user is a member of the group with access to the S3
128 Repeat the process again, creating a group for the Filepicker bucket and
129 creating a user with access to that group. These credentials will be passed
132 #### somewhat secure S3 access
133 Create two groups as described in the `secure S3 access` section above.
135 Create a single user, save the credentials as described in the
136 `insecure S3 access` section above, and pass the credentials on to Filepicker.
138 Add the single user to both groups.
140 This is less secure because if your web server or Filepicker get compromised
141 (so there are two points for potential failure), the single compromised
142 user has full access to both buckets.
144 ### Amazon Cloudfront CDN
145 [Cloudfront CDN](http://aws.amazon.com/cloudfront/) assists static file hosting.
148 [Amazon's instructions](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/GettingStarted.html)
149 to host static files out of the appropriate S3 bucket. Note that Django's static
150 file upload process has been modified to mark static files as publicly
153 In the settings for the Cloudfront Distribution, copy the "Domain Name" from
154 General settings and set `CLOUDFRONT_DOMAIN` to it. For example, `abcdefghij.cloudfront.net`.
156 ### Amazon Mechanical Turk
157 Mechanical turk is employed to generate human feedback from uploaded notes.
158 This service is helpful for generating flash cards and quizzes.
160 This service is optional and it might cause unexpected charges when
161 deployed. If the required environment variable is not found,
162 then no errors will occur and no mechanical turk tasks will be created, avoiding any unexpected
165 The `MTURK_HOST` environment variable is almost certainly
166 `"mechanicalturk.amazonaws.com"`.
168 The code will create and publish HITs on your behalf.
171 This software uses [Google Drive](https://developers.google.com/drive/) to
172 convert documents to and from various file formats.
174 A Google Drive service account with access to the Google Drive is required.
175 This may be done with a Google Apps account with administrative privileges, or ask
176 your business sysadmin.
178 Follow [Google's instructions](https://developers.google.com/drive/delegation)
179 to create a Google Drive service account.
181 Convert the p12 file into a Base64 encoded string for the
182 `GOOGLE_SERVICE_KEY_BASE64` environment variable. There are many ways to do
183 this. If Python is available, the
184 [binascii library](https://docs.python.org/2/library/binascii.html#binascii.b2a_base64)
185 makes this very easy:
188 with open('file.p12', 'r') as f:
189 print binascii.b2a_base64(f.read)
191 Copy the contents of `client_secret_*.apps.googleusercontent.com.json` into the
192 `GOOGLE_CLIENT_SECRETS` environment variable.
195 This software uses [Filepicker.io](https://www.inkfilepicker.com/) for uploading
196 files. This requires an account with Filepicker.
198 Filepicker can use an additional third party file hosting site where it may
199 send uploaded files. This project, in production, uses Amazon S3 as the third
200 party. See the Amazon S3 section above for more information.
202 Create a new App with Web SDK and provide the Heroku App URL for the
203 Application's URL. You'll be given an API Key for the App. Paste this into the
204 `FILEPICKER_API_KEY` environment variable.
206 Find the 'App Security' button on the left hand side of the web site. Make sure
207 'Use Security' is enabled. Generate a new secret key. Paste this key into the
208 `FILEPICKER_SECRET` environment variable.
210 If you have an upgraded plan, you can configure Filepicker to have access to
211 your Filepicker S3 bucket. Click 'Amazon S3' on the left hand side menu and
212 supply the credentials for the user with access to the Filepicker S3 bucket.
215 KarmaNotes uses IndexDen to create a searchable index of all the notes in the
216 system. Create an free IndexDen account at
217 [their homepage](http://indexden.com/). You will be given a private URL that
218 accesses your IndexDen account. This URL is visible on your dashboard (you
219 might need to scroll down).
221 Set the `INDEXDEN_PRIVATE_URL` environment variable to your private URL.
223 Set the `INDEXDEN_INDEX` environment variable to the name of the index you want
224 to use for KarmaNotes. The index will be created automatically when KarmaNotes
225 is run if it doesn't already exist. It may be created through the GUI if
230 Twitter is used to post updates about new courses. Access to the Twitter API
231 will be required for this task.
233 If this Twitter feature is desired, the consumer key and secret as well as the
234 access token key and secret are needed by the software.
236 If the required environment variables are not found, then no errors will occur
237 and no tweets will be posted.
240 [create a new Twitter application](https://dev.twitter.com/apps/new).
241 Use your Heroku App URL for the website field. Leave the Callback field blank.
243 Make sure this application has read/write access. Generate an access token. Go
244 to your OAuth settings, and grab the "Consumer key", "Consumer secret",
245 "Access token", and "Access token secret". Paste these, respectively, into the
246 environment variables `TWITTER_CONSUMER_KEY`, `TWITTER_CONSUMER_SECRET`,
247 `TWITTER_ACCESS_TOKEN_KEY`, `TWITTER_ACCESS_TOKEN_SECRET`.
251 If you wish to host your system publicly, you'll need an SSL certificate
252 signed by a proper authority.
254 Follow [Heroku's SSL setup](https://devcenter.heroku.com/articles/ssl-endpoint)
255 to get SSL running on your server.
257 You may set the `SSL_REDIRECT` environment variable to `true` to make KarmaNotes
258 redirect insecure connections to secure ones.
262 KarmaNotes is a Heroku application. Download the [Heroku toolbelt](https://toolbelt.heroku.com/).
264 Before your running it for the first time, there are
267 1. `source venv/bin/activate`
268 1. `pip install -r requirements.txt`
269 1. `pip install -r requirements-dev.txt`
270 1. `foreman run python manage.py syncdb --migrate --noinput`
271 1. `foreman run python manage.py createsuperuser`
272 1. `foreman run python manage.py fetch_usde_csv ./schools.csv`
273 1. `foreman run python manage.py import_usde _csv ./schools.csv`
274 1. `foreman run python manage.py sanitize_usde_schools`
276 To run KarmaNotes locally, make sure you are inside your
277 virtual environment (`source venv/bin/activate`) and run `foreman start`.
278 Press ctrl-C to kill foreman. Foreman will run Django's runserver command.
279 If you wish to have more control over how this is done, you can do
280 `foreman run python manage.py runserver <options>`. For running any other
281 `manage.py` commands, you should also precede them with `foreman run` like just shown.
282 This simply ensures that the environment variables from `.env` are present.
286 KarmaNotes is a Heroku application. Download the [Heroku toolbelt](https://toolbelt.heroku.com/).
288 To run KarmaNotes on Heroku, do `heroku create` and `git push heroku master` as typical
289 for a Heroku application. Set your the variable `BUILDPACK_URL` to
290 `https://github.com/charlesconnell/heroku-buildpack-karmanotes` to use a buildpack designed
291 to support KarmaNotes.
293 You will need to import the US Department of Education's list of accredited schools.
294 1. Fetch USDE schools with
295 `heroku run python manage.py fetch_usde_csv ./schools.csv`
296 1. Upload the schools into the database with
297 `heroku run python /manage.py import_usde _csv ./schools.csv`
298 1. Clean up redundant information with
299 `heroku run python /manage.py sanitize_usde_schools`
302 # Django Database management
306 We have setup Django to use
307 [south](http://south.aeracode.org/wiki/QuickStartGuide) for migrations. When
308 changing models, it is important to run
309 `foreman run python manage.py schemamigration` which will create a migration
310 to reflect the model changes into the database. These changes can be pulled
311 into the database with `foreman run python manage.py migrate`.
313 Sometimes the database already has a migration performed on it, but that
314 information wasn't told to south. There are subtleties to the process which
315 require looking at the south docs. As a tip, start by looking at the `--fake`
318 # Assets from Third Parties
320 A number of assets have been added to the repository which come from external
321 sources. It would be difficult to keep a complete list in this README and keep
322 it up to date. Software which originally came from outside parties can
323 generally be found in `karmaworld/assets`.
325 Additionally, all third party Python projects (downloaded and installed with
326 pip) are listed in these files:
329 * `requirements-dev.txt`
333 * KarmaNotes.org is a project of the FinalsClub Foundation with generous funding from the William and Flora Hewlett Foundation
335 * Also thanks to [rdegges](https://github.com/rdegges/django-skel) for the django-skel template