Merge pull request #98 from mrodrig/3.1.0

feature: add emptyFieldValue option

Author: mrodrig

README.md

@@ -42,14 +42,23 @@ Upgrading to v3 from v2? Check out the [upgrade guide](https://github.com/mrodri
 * `callback` - A function of the form `function (err, csv)`; 
   * This function will receive any errors and/or the string of CSV generated.
 * `options` - (Optional) A JSON document specifying any of the following key value pairs:
+  * `checkSchemaDifferences` - Boolean - Should all documents have the same schema?
+    * Default: `false`
+    * Note: An error will be thrown if some documents have differing schemas when this is set to `true`.
   * `delimiter` - Document - Specifies the different types of delimiters
     * `field` - String - Field Delimiter. 
       * Default: `,`
     * `wrap` - String - Wrap values in the delimiter of choice (e.g. wrap values in quotes). 
       * Default: `"`
     * `eol` - String - End of Line Delimiter. 
       * Default: `\n`
+  * `emptyFieldValue` - Any - Value that, if specified, will be substituted in for field values that are `undefined`, `null`, or an empty string.
+    * Default: none
   * `excelBOM` - Boolean - Should a unicode character be prepended to allow Excel to open a UTF-8 encoded file with non-ASCII characters present.
+  * `keys` - Array - Specify the keys (as strings) that should be converted. 
+    * Default: `null`
+    * If you have a nested object (ie. {info : {name: 'Mike'}}), then set this to ['info.name']
+    * If you want all keys to be converted, then specify ```null``` or don't specify the option to utilize the default.
   * `prependHeader` - Boolean - Should the auto-generated header be prepended as the first line in the CSV?
     * Default: `true`
   * `sortHeader` - Boolean - Should the header keys be sorted in alphabetical order? 
@@ -58,14 +67,6 @@ Upgrading to v3 from v2? Check out the [upgrade guide](https://github.com/mrodri
     * Default: `false`
   * `trimFieldValues` - Boolean - Should the field values be trimmed? (*in development*)
     * Default: `false`
-  * `checkSchemaDifferences` - Boolean - Should all documents have the same schema?
-    * Default: `false`
-    * Note: Change this to `false` if some documents are missing certain fields and you still want to convert the data.
-  * `keys` - Array - Specify the keys (as strings) that should be converted. 
-    * Default: `null`
-    * If you have a nested object (ie. {info : {name: 'Mike'}}), then set this to ['info.name']
-    * If you want all keys to be converted, then specify ```null``` or don't specify the option to utilize the default.
-    
 
 
 For examples, please refer to the [json2csv API Documentation (Link)](https://github.com/mrodrig/json-2-csv/wiki/json2csv-Documentation)
@@ -87,15 +88,15 @@ Available in version `2.2.0`, this functionality makes use of promises from the
     * `eol` - String - End of Line Delimiter. 
       * Default: `\n`
   * `excelBOM` - Boolean - Does the CSV contain a unicode character prepended in order to allow Excel to open a UTF-8 encoded file with non-ASCII characters present?
-  	* Default: `false`
-  * `trimHeaderFields` - Boolean - Should the header fields be trimmed? 
-    * Default: `false`
-  * `trimFieldValues` - Boolean - Should the field values be trimmed? 
     * Default: `false`
   * `keys` - Array - Specify the keys (as strings) that should be converted. 
     * Default: `null`
     * If you have a nested object (ie. `{info : {name: 'Mike'}}`), then set this to `['info.name']`
     * If you want all keys to be converted, then specify `null` or don't specify the option to utilize the default.
+  * `trimHeaderFields` - Boolean - Should the header fields be trimmed? 
+    * Default: `false`
+  * `trimFieldValues` - Boolean - Should the field values be trimmed? 
+    * Default: `false`
 
 For examples, please refer to the [csv2json API Documentation (Link)](https://github.com/mrodrig/json-2-csv/wiki/csv2json-Documentation)
 
@@ -109,18 +110,16 @@ Available in version `2.2.0`, this functionality makes use of promises from the
 $ npm test
 ```
 
-_Note_: This requires `mocha`, `should`, and `underscore`.
-
 To see test coverage, please run:
 ```bash
 $ npm run coverage
 ```
 
 Current Coverage is:
 ```
-Statements   : 100% ( 258/258 )
-Branches     : 100% ( 124/124 )
-Functions    : 100% ( 49/49 )
+Statements   : 100% ( 261/261 )
+Branches     : 100% ( 131/131 )
+Functions    : 100% ( 47/47 )
 Lines        : 100% ( 256/256 )
 ```
 
@@ -130,13 +129,12 @@ Please find the updated list (relocated to the Wiki) here: [Frequently Asked Que
 ## Features
 * Header Generation (per document keys)
 * Allows for conversion of specific keys in both json2csv and csv2json via the options.keys parameter (as of 1.1.2)
-* Verifies all documents have same schema (schema field order does not matter as of 1.1.0)
+* Document schema verification functionality (field order is irrelevant) (as of 1.1.0)
 * Supports sub-documents natively
 * Supports arrays as document values for both json2csv and csv2json
 * Custom ordering of columns (see F.A.Q. for more information)
 * Ability to re-generate the JSON documents that were used to generate the CSV (including nested documents)
 * Allows for custom field delimiters, end of line delimiters, etc.
-* Promisifiable via bluebird's .promisify(<function>) and .promisifyAll(<object>) (as of 1.1.1)
 * Wrapped value support for json2csv and csv2json (as of 1.3.0)
 * Support for multiple different schemas (as of 1.4.0)
 * Promisified versions of the functions are now available by default: json2csvAsync, csv2jsonAsync (as of 2.2.0)
@@ -145,3 +143,4 @@ Please find the updated list (relocated to the Wiki) here: [Frequently Asked Que
 	* `csv2json test.csv -o output.json`
 	* *and*
 	* `json2csv test.json -o output.csv -W -k arrayOfStrings -o output.csv`
+* Empty field value option (as of 3.1.0)

package.json

@@ -2,7 +2,7 @@
     "author": "mrodrig",
     "name": "json-2-csv",
     "description": "A JSON to CSV and CSV to JSON converter that natively supports sub-documents and auto-generates the CSV heading.",
-    "version": "3.0.0",
+    "version": "3.1.0",
     "repository": {
         "type": "git",
         "url": "http://github.com/mrodrig/json-2-csv.git"

src/json2csv.js

@@ -193,6 +193,11 @@ const Json2Csv = function(options) {
 
         fields.forEach((field) => {
             let recordFieldValue = path.evaluatePath(record, field);
+
+            if (!_.isUndefined(options.emptyFieldValue) && utils.isEmptyField(recordFieldValue)) {
+                recordFieldValue = options.emptyFieldValue;
+            }
+
             recordValues.push(recordFieldValue);
         });
 

src/utils.js

@@ -10,7 +10,8 @@ module.exports = {
     isDateRepresentation,
     computeSchemaDifferences,
     deepCopy,
-    convert
+    convert,
+    isEmptyField
 };
 
 /**
@@ -127,3 +128,13 @@ function computeSchemaDifferences(schemaA, schemaB) {
     return _.difference(schemaA, schemaB)
         .concat(_.difference(schemaB, schemaA));
 }
+
+/**
+ * Utility function to check if a field is considered empty so that the emptyFieldValue can be used instead
+ * @param fieldValue
+ * @returns {boolean}
+ */
+function isEmptyField(fieldValue) {
+    console.log(fieldValue);
+    return _.isUndefined(fieldValue) || _.isNull(fieldValue) || fieldValue === '';
+}

test/config/testCsvFilesList.js

@@ -23,7 +23,8 @@ const fs = require('fs'),
         {key: 'specifiedKeys', file: '../data/csv/specifiedKeys.csv'},
         {key: 'extraLine', file: '../data/csv/extraLine.csv'},
         {key: 'noHeader', file: '../data/csv/noHeader.csv'},
-        {key: 'sortedHeader', file: '../data/csv/sortedHeader.csv'}
+        {key: 'sortedHeader', file: '../data/csv/sortedHeader.csv'},
+        {key: 'emptyFieldValues', file: '../data/csv/emptyFieldValues.csv'}
     ];
 
 function readCsvFile(filePath) {

test/config/testJsonFilesList.js

@@ -16,5 +16,6 @@ module.exports = {
     trimmedFields: require('../data/json/trimmedFields'),
     trimHeader: require('../data/json/trimHeader'),
     trimmedHeader: require('../data/json/trimmedHeader'),
-    specifiedKeys: require('../data/json/specifiedKeys')
+    specifiedKeys: require('../data/json/specifiedKeys'),
+    emptyFieldValues: require('../data/json/emptyFieldValues')
 };

test/data/csv/emptyFieldValues.csv

@@ -0,0 +1,3 @@
+arrayOfStrings,object.subField,isBoolean,optionalField,number
+"[""test1"",""test2""]",,true,this one has it,
+"[""test3"",""test4""]",subValue,false,,7

test/data/json/emptyFieldValues.js

@@ -0,0 +1,16 @@
+module.exports = [
+    {
+        arrayOfStrings: ['test1', 'test2'],
+        object: { subField: '' },
+        number: undefined,
+        isBoolean: true,
+        optionalField: 'this one has it'
+    },
+    {
+        arrayOfStrings: ['test3', 'test4'],
+        object: { subField: 'subValue' },
+        number: 7,
+        isBoolean: false,
+        optionalField: null
+    }
+];

test/json2csv.js

@@ -343,6 +343,17 @@ function runTests(jsonTestData, csvTestData) {
                     keys: ['arrayOfStrings', 'object.subField']
                 });
             });
+
+            it('should use the specified empty field value, if provided', (done) => {
+                jsonTestData.emptyFieldValues[0].number = undefined;
+                converter.json2csv(jsonTestData.emptyFieldValues, (err, csv) => {
+                    if (err) done(err);
+                    csv.should.equal(csvTestData.emptyFieldValues);
+                    done();
+                }, {
+                    emptyFieldValue: ''
+                });
+            });
         });
     });
 

upgrade_guides/UPGRADE_2_to_3.md

@@ -12,7 +12,7 @@ available, however, you will need to update your code to use the new keys.
 The following lists the key name changes (left = old, right = new)
 
 ```
-EMPTY_FIELD_VALUE        --> * REMOVED *
+EMPTY_FIELD_VALUE        --> emptyFieldValue
 CHECK_SCHEMA_DIFFERENCES --> checkSchemaDifferences
 KEYS                     --> keys
 PARSE_CSV_NUMBERS        --> parseCsvNumbers
@@ -36,13 +36,6 @@ This was likely one of the least commonly used options and instead became a pain
 point since the array delimiter was required to be different from the field 
 delimiter prior to v3.0.0.
 
-* Empty Field Value Option Removed
-
-The empty field value option was previously not being used by the json2csv
-function, so for simplicity, it has been removed in v3.0.0. Please open an issue
-on the GitHub repository `mrodrig/json-2-csv` if you have a valid use case for
-this option to be implemented.
-
 * Schema Difference Check
 
 By default, the schema difference check for `json2csv` has now been disabled.