-
-
Notifications
You must be signed in to change notification settings - Fork 409
Description
Proposal
Under the existing design, user has to manually modify their csv string to match with csv reader defaults.
func (r *Rows) FromCSVString(s string) *Rows
function initialize a csv reader with defaults. The user may have a pipe separated csv or some other token. Also, the user cannot enable lazyQoutes (see: https://cs.opensource.google/go/go/+/refs/tags/go1.22.4:src/encoding/csv/reader.go;l=136) which is sometimes needed if you have a json string in the csv file without having to modify the csv extensively to match the defaults.
My proposal is to allow the user to define their csv reader freely with their own custom options.
Easiest way to do it without breaking existing code is the following:
func (r *Rows) FromCSVReader(csvReader *csv.Reader) *Rows {
for {
res, err := csvReader.Read()
if err != nil {
if errors.Is(err, io.EOF) {
break
}
panic(fmt.Sprintf("Parsing CSV string failed: %s", err.Error()))
}
row := make([]driver.Value, len(r.cols))
for i, v := range res {
row[i] = CSVColumnParser(strings.TrimSpace(v))
}
r.rows = append(r.rows, row)
}
return r
}
you can also check this commit that I pushed on my fork: stayingalivee@d0736ed
Use-cases
I added a test to demonstrate the use-case. see: stayingalivee@d0736ed#diff-335b77abfb8789fd51a279c7cbdbdae90b384df153ce1bbac40aa1468d5bbb89R472
for ease of access I will paste the test below:
func TestCSVRowParserFromCsvParser(t *testing.T) {
t.Parallel()
line := "a|NULL|[\"a\",\"b\"]"
r := strings.NewReader(line)
csvReader := csv.NewReader(r)
csvReader.Comma = '|'
csvReader.LazyQuotes = true
rs := NewRows([]string{"col1", "col2", "col3"}).FromCSVReader(csvReader)
db, mock, err := New()
if err != nil {
t.Fatalf("an error '%s' was not expected when opening a stub database connection", err)
}
defer db.Close()
mock.ExpectQuery("SELECT").WillReturnRows(rs)
rw, err := db.Query("SELECT")
if err != nil {
t.Fatalf("unexpected error: %s", err)
}
defer rw.Close()
var col1 string
var col2 []byte
var col3 string
var col3Json []string
rw.Next()
if err = rw.Scan(&col1, &col2, &col3); err != nil {
t.Fatalf("unexpected error: %s", err)
}
if col1 != "a" {
t.Fatalf("expected col1 to be 'a', but got [%T]:%+v", col1, col1)
}
if col2 != nil {
t.Fatalf("expected col2 to be nil, but got [%T]:%+v", col2, col2)
}
err = json.Unmarshal([]byte(col3), &col3Json)
if err != nil {
t.Fatalf("expected valid json, but got [%T]:%+v", col3, col3)
}
if !reflect.DeepEqual(col3Json, []string{"a", "b"}) {
t.Fatalf("expected col3Json to be [\"a\", \"b\"], but got [%T]:%+v", col3Json, col3Json)
}
}
If you find this helpful let me know so I can submit a pull request.