Merge pull request #22 from vsoch/fix/paper

updating get_userhome functions and putting paper lines, one per line

Author: vsoch

paper/paper.md

@@ -17,104 +17,68 @@ bibliography: paper.bib
 
 # Summary
 
-[WatchMe](https://vsoch.github.io/watchme/) is a simple tool to allow for reproducibly watching for changes in one or more 
-web pages, system resources, or any task function that is provided to the library.
-It addresses a problem in research that it's highly challenging to create and share
-reproducible tasks, meaning:
+[WatchMe](https://vsoch.github.io/watchme/) is a simple tool to allow for reproducibly watching for changes in one or more web pages, system resources, or any task function that is provided to the library.
+It addresses a problem in research that it's highly challenging to create and share reproducible tasks, meaning:
 
  1. a configuration file (recipe) stores the parameters for tasks including a function to run, a frequency, and any other necessary variables
  2. the tasks are automatically run at some frequency
  3. the results of the runs are saved automatically via version control
  4. the results collected can be re-assembled into temporal data structures that are ready for analysis
  5. the entire base (configuration, tasks, and results) can be shared via GitHub, and reproduced by others
 
-With WatchMe, a researcher can easily generate a repository (a watcher) that is configured
-to run one or more tasks at a particular frequency, and automatically commit changes to git.
-If he or she chooses, the repository can be pushed to a version control service like GitHub,
-and the entire configuration and set of tasks is easily reproducible by anyone that uses
-the client to get the repository. Each watcher uses git not only for version control of
-configuration files, but as a temporal database from which the results of the task runs can
-be extracted. Every change to a task within a watcher directory is also recorded via
-git, making the entire setup well documented with minimal to no work needed by the
-researcher.
+With WatchMe, a researcher can easily generate a repository (a watcher) that is configured to run one or more tasks at a particular frequency, and automatically commit changes to git.
+If he or she chooses, the repository can be pushed to a version control service like GitHub, and the entire configuration and set of tasks is easily reproducible by anyone that uses the client to get the repository. Each watcher uses git not only for version control of configuration files, but as a temporal database from which the results of the task runs can be extracted. 
+Every change to a task within a watcher directory is also recorded via git, making the entire setup well documented with minimal to no work needed by the researcher.
 
 ## Background
 
-Reproducible monitoring and data collection for an individual researcher is a challenging task. Typically,
-if a web page or system resource is to be monitored, the researcher must write custom scripts and
-extraction steps, and in the best case scenario, he or she uses version control for the scripts or final
-result. While many online services exist to watch for changes in one or more web pages
-(e.g., see https://visualping.io/ for an example service), these resources are problematic
-for research use. Specifically:
+Reproducible monitoring and data collection for an individual researcher is a challenging task. 
+Typically, if a web page or system resource is to be monitored, the researcher must write custom scripts and extraction steps, and in the best case scenario, he or she uses version control for the scripts or final result. 
+While many online services exist to watch for changes in one or more web pages (e.g., see https://visualping.io/ for an example service), these resources are problematic for research use. Specifically:
 
  1. It's typically the case that you will be charged for more than a few pages
  2. It's not appropriate for a research setting where you would want programmatic parsing
  3. The configuration of your watcher is not reproducible.
 
-Thus, WatchMe is ideal for the individual researcher that does not want to (or cannot)
-pay for a service, and wants to be able to share their monitoring tasks with other
-researchers, such as for a publication or similar. It also allows for collaborative data
-collection, as multiple users can run the equivalent task, have data exported named
-uniquely, and then submit a pull request to combine the data. 
-
+Thus, WatchMe is ideal for the individual researcher that does not want to (or cannot) pay for a service, and wants to be able to share their monitoring tasks with other researchers, such as for a publication or similar. 
+It also allows for collaborative data collection, as multiple users can run the equivalent task, have data exported named uniquely, and then submit a pull request to combine the data. 
 
 ## WatchMe Tasks
 
-By default, WatchMe comes with two task types intended to provide general templates
-for creating specific monitoring tasks.
+By default, WatchMe comes with two task types intended to provide general templates for creating specific monitoring tasks.
 
 ### Web Tasks
 
-It's a common need to want to retrieve content from the web, whether that be a request
-to get a page, a subset of a page, the download of a file, or a post to an application
-programming interface (API). These general tasks perform these operations, with customizations
-to control the url, how the response is parsed, headers and parameters, and the result written. 
-For example, the  general set of web tasks can be used to check a set of cities for changes to weather or climate,
-to monitor an API endpoint, track changes in prices of item(s) of interest, 
-download a file at some frequency, or watch a job board for changes. For details about
-setup and usage, see the [urls tasks](https://vsoch.github.io/watchme/watchers/urls/) 
-documentation.
-
+It's a common need to want to retrieve content from the web, whether that be a request to get a page, a subset of a page, the download of a file, or a post to an application programming interface (API). 
+These general tasks perform these operations, with customizations to control the url, how the response is parsed, headers and parameters, and the result written. 
+For example, the  general set of web tasks can be used to check a set of cities for changes to weather or climate, to monitor an API endpoint, track changes in prices of item(s) of interest, download a file at some frequency, or watch a job board for changes. 
+For details about setup and usage, see the [urls tasks](https://vsoch.github.io/watchme/watchers/urls/) documentation.
 
 ### System Tasks
 
-The psutils library of functions uses the Python Psutil [@psutil] set of functions
-to monitor system resources, sensors, and python environment. Given the naming of data outputs
-based on the host, if a second user forked the example repository
-and ran it on his or her host, he or she could open a pull request to contribute new data. 
-Given the unique naming of each task file, the data could co-exist with previous
-data generated on other hosts. Given the common export formats,
-common analyses could be shared and run on the exports by the different users.
-See the [psutils tasks](https://vsoch.github.io/watchme/watchers/psutils/) 
-documentation for details, and continue reading for a specific example.
+The psutils library of functions uses the Python Psutil [@psutil] set of functions to monitor system resources, sensors, and python environment. Given the naming of data outputs based on the host, if a second user forked the example repository and ran it on his or her host, he or she could open a pull request to contribute new data. 
+Given the unique naming of each task file, the data could co-exist with previous data generated on other hosts. 
+Given the common export formats, common analyses could be shared and run on the exports by the different users.
+See the [psutils tasks](https://vsoch.github.io/watchme/watchers/psutils/) documentation for details, and continue reading for a specific example.
 An example that uses the set of system tasks is discussed next.
 
 
 ## Research Usage
 
-The command line usage of watchme, along with making the tool programmatic, 
-also makes it ideal for usage on research clusters, or custom usage within scripts.
-Importantly, WatchMe is able to take a repository of result files produced by one
-or more contributors, and export data structures that keep a record of timestamps,
-results, and commit ids for each addition of a results file.
+The command line usage of watchme, along with making the tool programmatic, also makes it ideal for usage on research clusters, or custom usage within scripts.
+Importantly, WatchMe is able to take a repository of result files produced by one or more contributors, and export data structures that keep a record of timestamps, results, and commit ids for each addition of a results file.
 
 ### Watcher Example
 
-As an example, the repository [watchme-system](https://github.com/vsoch/watchme-system) runs a
-set of hourly tasks to measure the host networking, cpu and memory usage, sensors 
-(battery, temperature, fans), and other user and python-specific data. After installing
-watchme, a second researcher could easily obtain the task by doing:
+As an example, the repository [watchme-system](https://github.com/vsoch/watchme-system) runs a set of hourly tasks to measure the host networking, cpu and memory usage, sensors (battery, temperature, fans), and other user and python-specific data. 
+After installing watchme, a second researcher could easily obtain the task by doing:
 
 ```bash
 $ watchme get https://www.github.com/vsoch/watchme-system system
 ```
 
-The command above would clone the repository, check that it was a valid Watcher (indicated
-by presence of a configuration file named watchme.cfg) and then download the folder
-to a new watcher named "system" (the second argument) in the default Watchme base folder,
-located at $HOME/.watchme. The organization of any watcher is intuitive - the top level
-folder is the name for the watcher, and the folders inside that begin with "task-" 
-represent the various task folders:
+The command above would clone the repository, check that it was a valid Watcher (indicated by presence of a configuration file named watchme.cfg) and then download the folder to a new watcher named "system" (the second argument) in the default Watchme base folder, located at $HOME/.watchme. 
+The organization of any watcher is intuitive - the top level folder is the name for the watcher, and the folders inside that begin with "task-" represent the various task folders:
 
 ```bash
 /.watchme/system$ tree
@@ -145,14 +109,11 @@ represent the various task folders:
 └── watchme.cfg
 ```
 
-Notice that each task folder has a result file, along with a timestamp to indicate 
-when the watcher was last run. The user can edit the watchme.cfg if desired, or simply
-activate and schedule the watcher (optionally disabling a subset of tasks) to run 
-at some frequency (e.g., hourly) and commit to git. No further work is required 
-by the researcher other than keeping the host machine turned on. The researcher can push the results
-to a GitHub repository (as was done in this case) and at any time, export the results
-for a particular result file. In the command below, we use the "watchme" client to export the watcher folder "system"
-for a task called "task-memory". We ask the watcher to parse the result content as json: 
+Notice that each task folder has a result file, along with a timestamp to indicate  when the watcher was last run. 
+The user can edit the watchme.cfg if desired, or simply activate and schedule the watcher (optionally disabling a subset of tasks) to run at some frequency (e.g., hourly) and commit to git. 
+No further work is required by the researcher other than keeping the host machine turned on. 
+The researcher can push the results to a GitHub repository (as was done in this case) and at any time, export the results for a particular result file. 
+In the command below, we use the "watchme" client to export the watcher folder "system" for a task called "task-memory". We ask the watcher to parse the result content as json: 
 
 
 ```bash
@@ -204,37 +165,28 @@ $ watchme export system task-memory vanessa-thinkpad-t460s_vanessa.json --json
 }
 ```
 
-While only two commits are shown in the result above, an actual export for
-this particular watcher has results for memory metrics collected on the hour.
-The researcher could then perform an [analysis](https://github.com/vsoch/watchme-system/blob/master/data/watchme-task-analysis.ipynb) 
-using the data collected. As an example, here is a plot from such an analysis that
-tracks virtual memory usage of this author, recorded every hour,
- over two weekend days. 
+While only two commits are shown in the result above, an actual export for this particular watcher has results for memory metrics collected on the hour.
+The researcher could then perform an [analysis](https://github.com/vsoch/watchme-system/blob/master/data/watchme-task-analysis.ipynb) using the data collected. 
+As an example, here is a plot from such an analysis that tracks virtual memory usage of this author, recorded every hour, over two weekend days. 
 
 ![img/virtual-memory-used.png](img/virtual-memory-used.png)
 
-Interestingly, we can see a pattern that correlates with the activity of the author
-during the day. Virtual memory usage is low from the previous evening (1800 hours)
-through the early morning (0600 hours) and then rises sharply when the author starts
-to work. It goes down briefly in the early afternoon when the author pauses for a break,
-and picks up afterward, stopping when it's time for dinner. We see that the
-system's core temperature follows a similar trend:
+Interestingly, we can see a pattern that correlates with the activity of the author during the day. Virtual memory usage is low from the previous evening (1800 hours) through the early morning (0600 hours) and then rises sharply when the author starts to work. 
+It goes down briefly in the early afternoon when the author pauses for a break, and picks up afterward, stopping when it's time for dinner. 
+We see that the system's core temperature follows a similar trend:
 
 ![img/core-temp.png](img/core-temp.png)
 
 We also see that the computer was briefly unplugged after the morning work session.
 
 ![img/battery.png](img/battery.png)
 
+These kinds of metrics are interesting to answer research questions about system resources and behavior, and represent only the tip of the iceberg in terms of the scope of data that WatchMe could help collect. 
+For example, WatchMe would have interesting use cases for monitoring resources or jobs for HPC, or watching for changes in any kind of web resource (prices, climate data, API endpoints, etc.). 
+For other examples, see the [WatchMe Examples](https://vsoch.github.io/watchme/examples) page.
 
-These kinds of metrics are interesting to answer research questions about system
-resources and behavior, and represent only the tip of the iceberg in terms of the
-scope of data that WatchMe could help collect. For example, WatchMe would have interesting
-use cases for monitoring resources or jobs for HPC, or watching for changes in any
-kind of web resource (prices, climate data, API endpoints, etc.) For other examples, see the 
-[WatchMe Examples](https://vsoch.github.io/watchme/examples) page.
-
-More information on WatchMe, including examples, information on watcher tasks, and function documentation is provided at the WatchMe <a href="https://vsoch.github.io/watchme/" target="_blank">documentation</a>. Others are encouraged to give feedback, ask questions, and request
+More information on WatchMe, including examples, information on watcher tasks, and function documentation is provided at the WatchMe <a href="https://vsoch.github.io/watchme/" target="_blank">documentation</a>. 
+Others are encouraged to give feedback, ask questions, and request
 new task functions or examples on <a href="https://www.github.com/vsoch/watchme/issues" target="_blank">the issue board</a>.
 
 # References

watchme/tests/test_utils.py

@@ -11,6 +11,7 @@
 import tempfile
 import shutil
 import json
+from sys import platform
 import os
 
 
@@ -118,7 +119,11 @@ def test_userhome(self):
         from watchme.utils import get_user
         user = get_user()
         userhome = get_userhome()
-        self.assertEqual('/home/%s' % user, userhome)
+        print("Userhome is %s" % userhome)
+        if platform.startswith("linux"):
+            self.assertEqual('/home/%s' % user, userhome)
+        elif platform == "darwin":
+            self.assertEqual('/Users/%s' % user, userhome)
 
     def test_files(self):
         print('Testing utils.generate_temporary_files')

watchme/utils/fileio.py

@@ -12,17 +12,17 @@
 
 '''
 
-import configparser
 import errno
 import os
-import pwd
-import re
 import tempfile
 import json
 import io
 import socket
 import shutil
 import sys
+import getpass
+
+
 
 from watchme.logger import bot
 
@@ -31,11 +31,11 @@
 def get_userhome():
     '''get the user home based on the effective uid
     '''
-    return pwd.getpwuid(os.getuid())[5]
+    return os.path.expanduser("~")
 
 def get_user():
     '''return the active user'''
-    return os.path.basename(get_userhome())
+    return getpass.getuser()
 
 def get_host():
     '''return the hostname'''