Skip to content

Add FromCsvReader method to allow building rows with customized csv reader.Β #339

@stayingalivee

Description

@stayingalivee

Proposal

Under the existing design, user has to manually modify their csv string to match with csv reader defaults.
func (r *Rows) FromCSVString(s string) *Rows function initialize a csv reader with defaults. The user may have a pipe separated csv or some other token. Also, the user cannot enable lazyQoutes (see: https://cs.opensource.google/go/go/+/refs/tags/go1.22.4:src/encoding/csv/reader.go;l=136) which is sometimes needed if you have a json string in the csv file without having to modify the csv extensively to match the defaults.

My proposal is to allow the user to define their csv reader freely with their own custom options.
Easiest way to do it without breaking existing code is the following:

func (r *Rows) FromCSVReader(csvReader *csv.Reader) *Rows {
	for {
		res, err := csvReader.Read()
		if err != nil {
			if errors.Is(err, io.EOF) {
				break
			}
			panic(fmt.Sprintf("Parsing CSV string failed: %s", err.Error()))
		}

		row := make([]driver.Value, len(r.cols))
		for i, v := range res {
			row[i] = CSVColumnParser(strings.TrimSpace(v))
		}
		r.rows = append(r.rows, row)
	}
	return r
}

you can also check this commit that I pushed on my fork: stayingalivee@d0736ed

Use-cases

I added a test to demonstrate the use-case. see: stayingalivee@d0736ed#diff-335b77abfb8789fd51a279c7cbdbdae90b384df153ce1bbac40aa1468d5bbb89R472

for ease of access I will paste the test below:

func TestCSVRowParserFromCsvParser(t *testing.T) {
	t.Parallel()

	line := "a|NULL|[\"a\",\"b\"]"

	r := strings.NewReader(line)
	csvReader := csv.NewReader(r)
	csvReader.Comma = '|'
	csvReader.LazyQuotes = true

	rs := NewRows([]string{"col1", "col2", "col3"}).FromCSVReader(csvReader)
	db, mock, err := New()
	if err != nil {
		t.Fatalf("an error '%s' was not expected when opening a stub database connection", err)
	}
	defer db.Close()

	mock.ExpectQuery("SELECT").WillReturnRows(rs)

	rw, err := db.Query("SELECT")
	if err != nil {
		t.Fatalf("unexpected error: %s", err)
	}
	defer rw.Close()
	var col1 string
	var col2 []byte
	var col3 string
	var col3Json []string

	rw.Next()
	if err = rw.Scan(&col1, &col2, &col3); err != nil {
		t.Fatalf("unexpected error: %s", err)
	}
	if col1 != "a" {
		t.Fatalf("expected col1 to be 'a', but got [%T]:%+v", col1, col1)
	}
	if col2 != nil {
		t.Fatalf("expected col2 to be nil, but got [%T]:%+v", col2, col2)
	}

	err = json.Unmarshal([]byte(col3), &col3Json)
	if err != nil {
		t.Fatalf("expected valid json, but got [%T]:%+v", col3, col3)
	}

	if !reflect.DeepEqual(col3Json, []string{"a", "b"}) {
		t.Fatalf("expected col3Json to be [\"a\", \"b\"], but got [%T]:%+v", col3Json, col3Json)
	}
}

If you find this helpful let me know so I can submit a pull request.

Metadata

Metadata

Assignees

No one assigned

    Labels

    good-first-issuegreat issue for folks who are first time contributors to this repo

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions