Auto refresh token

README.md

@@ -9,10 +9,8 @@ Borrows liberally from @anthonywu's [Strava API Experiment](https://github.com/a
 1. **Get data from Runkeeper**<br>Next, you need to **get your data from Runkeeper.** Go to the Settings page, and look for "Export Data" near the bottom. Define your time range, wait a minute or two, and then click download. Unzip the file - the directory should have .gpx files for all of your GPS-tracked runs, and two spreadsheets - "measurements.csv" and "cardio_activities.csv".
 1. **Navigate to folder**<br>Open a shell (accessed by using the "Terminal" application on MacOS) and `cd` to the data directory (the folder you just downloaded - should be something like "runkeeper-data-export-1234567").
 1. **Install requirements**<br>Install the requirements - from any shell run `pip install -r requirements.txt`
-1. **Get authorization from Strava**<br>Next, we need to **get an Authorization Token from Strava** for your Athlete account. Run the command `python strava_local_client.py get_write_token <client_id> <client_secret>` where you replace `<client_id>` and `<client_secret>` with the codes you pulled from the [Strava API Management Page](https://www.strava.com/settings/api). It should open a browser and ask you to log in to Strava. You should then be shown a code - copy this, and either:
-	1. (Preferably) Set the environment variable `STRAVA_UPLOADER_TOKEN` before running `uploader.py`, e.g. `STRAVA_UPLOADER_TOKEN=my_token ./uploader.py`
-	1. Or paste it in the `uploader.py` file as the `access_token` variable, replacing `None`. Don't forget to quote the variable, e.g. `access_token = "my_token"`
-1. **Upload to Strava**<br>Now we're ready to upload. Run `STRAVA_UPLOADER_TOKEN=my_token ./uploader.py` and let it run!
+1. **Get authorization from Strava**<br>Next, we need to **get an Authorization Token from Strava** for your Athlete account. Run the command `python strava_local_client.py get_write_token <client_id> <client_secret>` where you replace `<client_id>` and `<client_secret>` with the codes you pulled from the [Strava API Management Page](https://www.strava.com/settings/api). It should open a browser and ask you to log in to Strava. You should then be shown a code, which is automatically saved with other data in a file called `tokens.txt` in the current directory. This is used in the next step.
+1. **Upload to Strava**<br>Now we're ready to upload. Run `./uploader.py` and let it run! Make sure `tokens.txt` is in the current directory.
 
 **A few notes on how this works:**
 - The script will crawl through the cardio activities csv file line for line, uploading each event.
@@ -22,10 +20,12 @@ Borrows liberally from @anthonywu's [Strava API Experiment](https://github.com/a
 - It will move successfully uploaded GPX files to a sub-folder called archive.
 - It will try to catch various errors, and ignore duplicate files.
 - It will log everything in a file `strava-uploader.log`.
+- If the program quits because it hit the daily upload limit, you can simply run `./uploader.py` again the next day (perhaps as a cron job) to continue. Needed tokens are refreshed and stored in the `tokens.txt` file.
 
 ## Misc other notes:
 - Do NOT modify or even save (without modification) the CSV from Excel. Even if you just open it and save it with no modification, Excel changes the date formatting which will break this script. If you do need to modify the CSV for some reason (e.g., mine had a run with a missing distance, not clear why), do it in Sublime or another text editor.
 - I personally ran into a few errors of "malformed GPX files". You can try opening the file in a text editor and looking for issues - look for missing closure tags (e.g., `</trkseg>`) - that was the issue with one of my files. You could also try to use other solutions - some ideas that solved other issues [here](https://support.strava.com/hc/en-us/articles/216942247-How-to-Fix-GPX-File-Errors).
+
 ## Updates specific to this branch
 
 You can use this script to upload a non-Runkeeper file in CSV format.  The current Runkeeper CSV file format includes the following columns: Activity Id, Date,Type, Route Name, Distance (mi), Duration, Average Pace, Average Speed (mph), Calories Burned, Climb (ft), Average Heart Rate (bpm), Friend's Tagged, Notes, GPX File.  If you wish to upload a non-Runkeeper file you have to create a cardioActivities.csv in this folder containing at least the following columns: Activity Id, Date, Type, Distance (mi), Duration.  The non-Runkeeper file must have matching column names to the Runkeeper original!  The GPX file if included should be a filename located in the same folder.

strava_local_client.py

@@ -16,6 +16,7 @@
 
 import stravalib
 from flask import Flask, request
+import os
 
 app = Flask(__name__)
 
@@ -25,16 +26,28 @@
 CLIENT_ID = None
 CLIENT_SECRET = None
 
+TOKEN_FILE = "tokens.txt"
+
 @app.route("/auth")
 def auth_callback():
     code = request.args.get('code')
     access_token = API_CLIENT.exchange_code_for_token(
         client_id=CLIENT_ID,
         client_secret=CLIENT_SECRET,
         code=code
-        )
-    return access_token['access_token']
+    )
+
+    if os.path.isfile(TOKEN_FILE):
+        os.rename(TOKEN_FILE, TOKEN_FILE + '.bak')
 
+    with open(TOKEN_FILE, 'w') as output:
+        output.write('client' + '\t' + CLIENT_ID + '\n')
+        output.write('secret' + '\t' + CLIENT_SECRET + '\n')
+        output.write('refresh' + '\t' + access_token['refresh_token'] + '\n')
+        output.write('access' + '\t' + access_token['access_token'] + '\n')
+        output.write('expiry' + '\t' + str(access_token['expires_at']) + '\n')
+
+    return access_token['access_token']
 
 if __name__ == '__main__':
     import docopt

uploader.py

@@ -17,15 +17,10 @@
 # Access Token
 #
 # You need to run the strava_local_client.py script, with your application's ID and secret,
-# to generate the access token.
-#
-# When you have the access token, you can
-#   (a) set an environment variable `STRAVA_UPLOADER_TOKEN` or;
-#   (b) replace `None` below with the token in quote marks, e.g. access_token = 'token'
+# to generate the access and other tokens in the following token_file.
 #####################################
-access_token = None
-
 cardio_file = 'cardioActivities.csv'
+token_file = 'tokens.txt'
 
 archive_dir = 'archive'
 skip_dir = 'skipped'
@@ -63,35 +58,85 @@ def get_cardio_file():
 	logger.error(cardio_file + ' file cannot be found')
 	exit(1)
 
-def get_strava_access_token():
-	global access_token
+def write_strava_access_tokens(tokens):
+	if os.path.isfile(token_file):
+		os.rename(token_file, token_file + '.bak')
 
-	if access_token is not None:
-		logger.info('Found access token')
-		return access_token
+	with open(token_file, 'w') as output:
+		for key, value in tokens.items():
+			output.write(key + '\t' + str(value) + '\n')
 
-	access_token = os.environ.get('STRAVA_UPLOADER_TOKEN')
-	if access_token is not None:
-		logger.info('Found access token')
-		return access_token
 
-	logger.error('Access token not found. Please set the env variable STRAVA_UPLOADER_TOKEN')
-	exit(1)
+def refresh_strava_access_tokens(client):
+	logger.info('Refreshing Strava access tokens')
+
+	tokens = client.tokens
+
+	refresh_response = client.refresh_access_token(
+		client_id=tokens['client'],
+		client_secret=tokens['secret'],
+		refresh_token=tokens['refresh'])
+
+	tokens['access'] = refresh_response['access_token'] # this is a short-lived access token
+	tokens['refresh'] = refresh_response['refresh_token']
+	tokens['expiry'] = refresh_response['expires_at']
+
+	client.access_token = tokens['access']
+	client.token_expires_at = int(tokens['expiry']) - 20*60 # update 20 mins before actual expiry
+
+	write_strava_access_tokens(tokens)
+
+	logger.info('Refreshed Strava access token expires at local time ' + time.asctime(time.localtime(tokens['expiry'])))
+
+	client.tokens = tokens
+	return client
+
+
+def get_strava_access_token(client):
+
+	tokens = dict()
+
+	if os.path.isfile(token_file):
+		with open(token_file, "r") as token_input:
+			for linein in token_input:
+				line = linein.split()
+				token = line[0].casefold()
+				tokens[token] = line[1]
+
+	if 'client' not in tokens or 'secret' not in tokens or 'refresh' not in tokens:
+		logger.info('Need client, secret and refresh tokens in the ' + token_file + ' file')
+		exit(1)
+
+	if 'access' in tokens:
+		client.access_token = tokens['access']
+	if 'expiry' in tokens:
+		client.token_expires_at = int(tokens['expiry']) - 20*60 # update 20 mins before actual expiry
+	else:
+		client.token_expires_at = 0 # force refrech
+
+	client.tokens = tokens
+
+	if int(time.time()) > client.token_expires_at:
+		client = refresh_strava_access_tokens(client)
+
+	return client
 
 def get_strava_client():
-	token = get_strava_access_token()
+
 	rate_limiter = RateLimiter()
 	rate_limiter.rules.append(XRateLimitRule(
 			{'short': {'usageFieldIndex': 0, 'usage': 0,
 						 # 60s * 15 = 15 min
-						 'limit': 100, 'time': (60*15),
-						 'lastExceeded': None,},
+						 'limit': 100, 'time': (60 * 15),
+						 'lastExceeded': None, },
 			 'long': {'usageFieldIndex': 1, 'usage': 0,
 						# 60s * 60m * 24 = 1 day
-						'limit': 1000, 'time': (60*60*24),
+						'limit': 1000, 'time': (60 * 60 * 24),
 						'lastExceeded': None}}))
+
 	client = Client(rate_limiter=rate_limiter)
-	client.access_token = token
+	client = get_strava_access_token(client)
+
 	return client
 
 def archive_file(file):
@@ -163,8 +208,7 @@ def upload_gpx(client, gpxfile, strava_activity_type, notes):
 			if i > 0:
 				logger.error("Daily Rate limit exceeded - exiting program")
 				exit(1)
-			logger.warning("Rate limit exceeded in uploading - pausing uploads for 15 minutes to avoid rate-limit")
-			time.sleep(900)
+			wait_for_timeout(client)
 			continue
 		except ConnectionError as err:
 			logger.error("No Internet connection: {}".format(err))
@@ -182,9 +226,7 @@ def upload_gpx(client, gpxfile, strava_activity_type, notes):
 			if i > 0:
 				logger.error("Daily Rate limit exceeded - exiting program")
 				exit(1)
-			logger.warning(
-				"Rate limit exceeded in processing upload - pausing uploads for 15 minutes to avoid rate-limit")
-			time.sleep(900)
+			wait_for_timeout(client)
 			continue
 		except exc.ActivityUploadFailed as err:
 			errStr = str(err)
@@ -286,6 +328,17 @@ def miles_to_meters(miles):
 def km_to_meters(km):
 	return float(km) * 1000
 
+def wait_for_timeout(client):
+	sleeptime = 15 # minutes
+
+	logger.warning("Rate limit exceeded - pausing transactions for " + str(sleeptime) + " minutes to avoid rate-limit")
+	time.sleep(sleeptime*60)
+
+	if int(time.time()) > client.token_expires_at:
+		client = refresh_strava_access_tokens(client)
+
+	return client
+
 def main():
 	set_up_logger()
 
@@ -301,8 +354,7 @@ def main():
 			if i > 0:
 				logger.error("Daily Rate limit exceeded - exiting program")
 				exit(1)
-			logger.warning("Rate limit exceeded in connecting - Retrying strava connection in 15 minutes")
-			time.sleep(900)
+			wait_for_timeout(client)
 			continue
 		break