Merge pull request #32 from sandialabs/restructure_modules

Restructure modules

Author: MichaelHopwood

NOTICE/statsmodels_LICENSE

@@ -0,0 +1,34 @@
+Copyright (C) 2006, Jonathan E. Taylor
+All rights reserved.
+
+Copyright (c) 2006-2008 Scipy Developers.
+All rights reserved.
+
+Copyright (c) 2009-2018 statsmodels Developers.
+All rights reserved.
+
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+  a. Redistributions of source code must retain the above copyright notice,
+     this list of conditions and the following disclaimer.
+  b. Redistributions in binary form must reproduce the above copyright
+     notice, this list of conditions and the following disclaimer in the
+     documentation and/or other materials provided with the distribution.
+  c. Neither the name of statsmodels nor the names of its contributors
+     may be used to endorse or promote products derived from this software
+     without specific prior written permission.
+
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ARE DISCLAIMED. IN NO EVENT SHALL STATSMODELS OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
+DAMAGE.

docs/whatsnew.rst

@@ -6,4 +6,7 @@ What's New
 
 These are new features and improvements of note in each release.
 
-.. include:: whatsnew/beta.rst
+
+.. include:: whatsnew/beta.rst
+
+.. include:: whatsnew/alpha.rst

docs/whatsnew/alpha.rst

@@ -0,0 +1,28 @@
+.. _whatsnew_alpha:
+
+First push
+-----------------------
+
+The original release of pvOps consists mostly of new features.
+
+New features
+~~~~~~~~~~~~
+
+* `text` module added which conducts natural language processing on Operations & Maintenance (O&M) tickets, or other.
+* `text2time` module investigates the relationship between the production timeseries data and the O&M tickets.
+* `timeseries` module conducts timeseries preprocessing and modeling
+* `iv` incorporates the ability to simulate current-voltage (IV) curves under different environmental, load, and failure conditions.
+
+
+Documentation
+~~~~~~~~~~~~~
+
+* Built original website
+* Add whatsnew
+* Add jupyter notebook embeddings
+
+Testing
+~~~~~~~
+
+* Built comprehensive tests with pytest 
+* Connected tests to automated testing pipeline

docs/whatsnew/beta.rst

@@ -3,26 +3,22 @@
 beta 
 -----------------------
 
-The beta release of pvOps will consist mostly of new features.
+New features and bug fixes are predominant in the beta versions.
 
 New features
 ~~~~~~~~~~~~
 
-* `text` module added which conducts natural language processing on Operations & Maintenance (O&M) tickets, or other.
-* `text2time` module investigates the relationship between the production timeseries data and the O&M tickets.
-* `timeseries` module conducts timeseries preprocessing and modeling
-* `iv` incorporates the ability to simulate current-voltage (IV) curves under different environmental, load, and failure conditions.
+* IV trace classification framework built according to literature (PR #25)
+* Timeseries IV simulation for highly customizable degradation of system parameters (PR #28)
+* Leverage pvlib solarposition package to populate content per site (PR #32)
+* Add coefficient-level evaluations linear models (PR #32)
+* Give user ability to input own test-train splits to linear modeller (PR #32)
+* Remap attributes function must retain the unaltered attributes (PR #32)
+* Interpolate O&M data onto production data where overlaps exist (PR #32)
 
+Bug fixes
+~~~~~~~~~
 
-Documentation
-~~~~~~~~~~~~~
-
-* Built original website
-* Add whatsnew
-* Add jupyter notebook embeddings
-
-Testing
-~~~~~~~
-
-* Built comprehensive tests with pytest 
-* Connected tests to automated testing pipeline
+* Basic package fixes to README (PR #27) and documentation configuration (PR #24)
+* Fix IV simulator bug for edge case where two IV curves added have equal I_{sc} (PR #30)
+* Neural network configuration referencing in 1D CNN (PR #32)

examples/example_data/remappings_asset.csv

@@ -1,28 +1,29 @@
 in,out_
-Inverter,inverter
-Recloser,recloser
-Transformer,transformer
-Switchgear,switchgear
-Combiner,combiner
+inverter,inverter
+recloser,recloser
+transformer,transformer
+switchgear,switchgear
+combiner,combiner
+substation,substation
+facility,facility
+energy meter,energy meter
+relay,relay
+met station,met station
+tracker,tracker
+module,module
 DC Disconnect,combiner
 Recombiner,combiner
 Feeder (Dip Pole/Array),wiring
 Ground-Mount PV System,module
-Module,module
-Tracker,tracker
-Met station,met station
 Weather Station,met station
 Pyranometer,met station
 Temperature sensor,met station
 Met station battery,met station
 Anemometer,met station
 Reference cell,met station
 Relative humidity sensor,met station
-Relay,relay
-Energy Meter,energy meter
 Meter,energy meter
 Energy Storage/Battery,energy storage
-Facility,facility
 AC Combiner,combiner
 Battery (Solar + storage facilities),energy storage
 Block,transformer
@@ -49,7 +50,6 @@ Site,other
 String,other
 String Inverter,inverter
 Subarray,other
-Substation,substation
 Summary,other
 Tracker control unit,tracker
 Tracking System,tracker

examples/example_data/remappings_response.csv

@@ -0,0 +1,21 @@
+in,out_
+Remote Troubleshooting,Remote troubleshotting/Reset
+Remote Reset,Remote troubleshotting/Reset
+13 - Remote Reset,Remote troubleshotting/Reset
+Power Cycle,Troubleshoot/Reset
+07 - Service,Troubleshoot/Reset
+09 - Inspection,Troubleshoot/Reset
+09-Inspection,Troubleshoot/Reset
+06 - Check,Troubleshoot/Reset
+08 - Test,Troubleshoot/Reset
+Clear faults,Troubleshoot/Reset
+Faults cleared,Troubleshoot/Reset
+Fault cleared manually,Troubleshoot/Reset
+No Action Required,Self-Resolved
+Problem Self-Resolved,Self-Resolved
+14 - Self Resolved,Self-Resolved
+Hardware Replacement,Replacement
+Replacement,Replacement
+01 - Replace,Replacement
+Fuse Replacement,Replacement
+Replace/Repair,Replacement

examples/text_class_example.py

@@ -43,7 +43,7 @@ def __init__(self, df, LABEL_COLUMN):
         self.df = df
 
     def summarize_text_data(self, DATA_COLUMN):
-        utils.summarize_text_data(self.df, DATA_COLUMN)
+        nlp_utils.summarize_text_data(self.df, DATA_COLUMN)
 
     def visualize_attribute_timeseries(self, DATE_COLUMN):
         col_dict = {"date": DATE_COLUMN, "label": self.LABEL_COLUMN}
@@ -102,6 +102,8 @@ def prep_data_for_ML(
         EVENTSTART_COLUMN,
         SAVE_DATA_COLUMN="CleanDesc",
         SAVE_DATE_COLUMN="ExtractedDates",
+        input_lst_add_stopwords=None,
+        input_lst_keep_stopwords=None,
     ):
         self.DATA_COLUMN = DATA_COLUMN
         self.EVENTSTART_COLUMN = EVENTSTART_COLUMN
@@ -115,13 +117,23 @@ def prep_data_for_ML(
                 self.df = self.df[self.df[self.LABEL_COLUMN] != lbl]
         self.df = self.df.sample(frac=1)
 
+        lst_add_stopwords = ["dtype", "say", "new",
+                             "length", "object", "u", "ha", "wa"]
+        lst_keep_words = ["new"]
+        if not isinstance(input_lst_add_stopwords, type(None)):
+            lst_add_stopwords += input_lst_add_stopwords
+        if not isinstance(input_lst_keep_stopwords, type(None)):
+            lst_keep_words += input_lst_keep_stopwords
+
         lst_stopwords = nlp_utils.create_stopwords(
             ["english"],
-            lst_add_words=["dtype", "say", "new",
-                           "length", "object", "u", "ha", "wa"],
-            lst_keep_words=["new"],
+            lst_add_words=lst_add_stopwords,
+            lst_keep_words=lst_keep_words,
         )
 
+        # Save so that can be easily referenced
+        self.lst_stopwords = lst_stopwords
+
         col_dict = {
             "data": DATA_COLUMN,
             "eventstart": EVENTSTART_COLUMN,
@@ -140,9 +152,8 @@ def prep_data_for_ML(
         self.DATA_COLUMN = SAVE_DATA_COLUMN
         return self.df[[DATA_COLUMN, SAVE_DATA_COLUMN]]
 
-    def visualize_freqPlot(self, LBL_CAT=None, DATA_COLUMN="CleanDesc", 
+    def visualize_freqPlot(self, LBL_CAT=None, DATA_COLUMN="CleanDesc",
                            **graph_aargs):
-
         if LBL_CAT is None:
             words = " ".join(self.df[DATA_COLUMN].tolist())
             num_rows = len(self.df[DATA_COLUMN].index)
@@ -269,15 +280,15 @@ def classify_supervised(
         user_defined_search_space=None,
         verbose=0,
     ):
-        """A wrapper function which evaluates the performance of many 
+        """A wrapper function which evaluates the performance of many
         supervised classifiers
 
             embedding : str
-                Definition of document embedding strategy with "tfidf" and 
+                Definition of document embedding strategy with "tfidf" and
                 "doc2vec" options
 
             setting : str
-                Thoroughness of supervised classification investigation with 
+                Thoroughness of supervised classification investigation with
                 following options:
                 'normal': a smaller subset of settings used in grid search
                 'detailed': a comprehensive set of settings used in grid search