prepare 5.10.1 release (#80)

* make the SDK a module

* go mod tidy

* fix bug where nil references are passed to persistent stores (#241)

* remove support for indirect patch in stream

* (5.0) complete test coverage for StreamProcessor

* turn on coverage enforcement

* fix logic for disabling db packages in coverage

* move some internal code around for clearer organization

* make sharedtest package internal, put public test helpers in new package

* DRY

* comment about data store tests

* use latest public go-sdk-common, go-sdk-events, go-server-sdk-evaluation

* remove unused polling logic for indirect/patch

* update go-server-sdk-evaluation for data model changes

* rm unnecessary type aliases that broke tests

* fix test that used an overly sensitive equality test for a flag without variations

* another test issue just like the previous commit

* move more test helpers + misc fixes

* test coverage

* move data store test suite to subpackage

* (5.0) fix bug in deleted item serialization

* more test coverage

* (5.0) interface cleanup: move data store types out of the way

* dependency fixes

* (5.0 - #7) updates for go-sdk-events API changes (#251)

* (5.0) miscellaneous doc comment improvements

* fix remaining godoc.org links

* custom badge for pkg.go.dev

* fix go.dev badge

* add API docs link under Learn More

* update dependencies

* use test sandboxing to validate persistent store test failure modes

* lint (use more general interface)

* (5.0) specify proxy URL as a string

* fix test

* fix test

* fix another test

* 5.0.0-beta.1

* add client decorator that temporarily disables events

* add scoped evaluation without events

* remove all database subpackages (being moved to separate projects) (#256)

* bump dependency versions and use ldlogtest package

* lint

* code example

* move AllFlagsState-related types out of main package, add builder

* rm obsolete references to HTTPOptions setter

* minor Makefile fix

* add clientSideAvailability to flag model

* prepare 4.17.3 release (#36)

* Releasing version 4.17.3

* (v5) use new client-side availability fields (#262)

* ensure SDK non-test code has no module-only dependencies (#263)

* 5.0.0-beta.6

* update redis repo name in comments

* use optional int type

* misc cleanup

* update dependencies

* add Go 1.15 CI job, use newer CircleCI images

* typo

* fix CI working directory

* fix CI syntax

* update readme to mention Go 1.15

* drop Go 1.13 build

* use go-server-sdk-evaluation 1.0.0-beta.5

* (#2) add test data source (#266)

* logging fixes/standardization

* use non-beta packages, remove prerelease info

* update release metadata

* make Releaser create release tags with and without the "v" prefix

* update to go-server-sdk-evaluation 1.0.1 for "less omitempty in flag JSON" fix (#272)

* fix comment typos

* serialize deleted item placeholders with full set of properties (#274)

* update go-sdk-common to 2.0.1

* bump eventsource to 1.6.2 for ch95617 logging fix (#275)

* use new jsonstream API (#277)

* don't require path property in put event

* update alpha dependencies

* update for go-jsonstream API changes

* build in Windows with Go 1.14

* bump go-sdk-common version to get user JSON parsing fix

* remove excessive logging in ldfilewatch, add log message for reloading file

* fail fast if SDK key has invalid characters, don't log the key

* bump go-server-sdk-evaluation to get semver parsing fix

* remove spurious logging of stream data

* merge in alias event support

* Removed the guides link

* update to go-server-sdk-evaluation 1.1.2 for ch100737 fix

* (#1) add unbounded segments interfaces, status provider, test suite (#288)

* add unbounded segments interfaces, status provider, test suite

* lint

* add package comment

* better component organization

* rename cache time to poll interval, improve tests

* rename segment keys to segment refs in unbounded segments API

* fix naming some more

* (#2) simplify(?) broadcasters using reflection (#289)

* (#3) misc cleanup of data store test suites (#290)

* (#4) finish adding support for big ("unbounded") segment in evaluations (#291)

* (#5) change "unbounded segments" to "big segments" + use latest prerelease packages (#293)

* fix segment ref string format in tests, update evaluator version

* Point at exp-alloc feature branch of -common and -evaluation (#297)

This just repoints the Go SDK at the exp-alloc feature branch of the go-sdk-common-private and go-server-sdk-evaluation-private repos.

* expose big segment store wrapper component for reuse by Relay

* lint

* add DuplicateKeysHandling option to ldfiledata (#299)

* use latest prerelease tags of go-sdk-common, go-server-sdk-evaluation

* use prerelease tags of go-sdk-common and go-server-sdk-evaluation

* update dependencies for JSON number parsing bugfix

* use go-sdk-common 2.3.0, go-server-sdk-evaluation 1.3.0

* add big segments control methods for use by Relay

* rm unused

* update go-server-sdk-evaluation to 1.4.1 for big segments bugfix

* use Releaser v2 config, add Go 1.16 CI

* rename big segments to Big Segments

* remove "non-module-build" CI test which can no longer be run

* add CI job for Go 1.17; update linter & test coverage script

* Update pull request template to correct CONTRIBUTING.md link

* Remove reference to dep in CONTRIBUTING.md (#310)

* SC-128558: Allow users to set custom headers on requests (#308)

* Add ability to set custom headers via Headers method

* better tests for simple proxy configuration (#311)

* update evaluation engine to fix circular prereqs issue, add more eval error logging

* add ServiceEndpoints to configuration API (#314)

* fix setting of trackEvents/trackReason in AllFlagsState data when there's an experiment (#315)

* Go SDK test service for use with the SDK test harness (#312)

* fix AllFlagsState DetailsOnlyForTrackedFlags behavior

* fix reason logic

* enable big segments contract tests (#317)

* implement tags

* update doc comments to match latest spec

* more doc comment updates

* support new contract tests that need to set service endpoints independently

* fix URI override logic

* add VariationForAll/VariationForAllIndex in test data builder, deprecate old names

* enforce 64-char limit on tag values

* enable contract tests for polling mode

* prevent data race on data source initialized property

* fix linter command

* update go-sdk-common to 2.5.1 for empty ldvalue.Raw fix

* enable contract tests for SecureModeHash

* Big Segment store tests should not expect an error value for "not found" condition

* handle error gracefully if http.NewRequest fails

Co-authored-by: Eli Bishop <[email protected]>
Co-authored-by: LaunchDarklyCI <[email protected]>
Co-authored-by: LaunchDarklyCI <[email protected]>
Co-authored-by: Harpo Roeder <[email protected]>
Co-authored-by: Ben Woskow <[email protected]>
Co-authored-by: Sam Stokes <[email protected]>
Co-authored-by: LaunchDarklyReleaseBot <[email protected]>
Co-authored-by: Casey Waldren <[email protected]>
Co-authored-by: Louis Chan <[email protected]>

Author: 9 people

internal/datasource/requestor.go

@@ -1,6 +1,7 @@
 package datasource
 
 import (
+	"fmt"
 	"io/ioutil"
 	"net/http"
 
@@ -83,6 +84,10 @@ func (r *requestorImpl) requestAll() ([]ldstoretypes.Collection, bool, error) {
 func (r *requestorImpl) makeRequest(resource string) ([]byte, bool, error) {
 	req, reqErr := http.NewRequest("GET", endpoints.AddPath(r.baseURI, resource), nil)
 	if reqErr != nil {
+		reqErr = fmt.Errorf(
+			"unable to create a poll request; this is not a network problem, most likely a bad base URI: %w",
+			reqErr,
+		)
 		return nil, false, reqErr
 	}
 	url := req.URL.String()

internal/datasource/streaming_data_source.go

@@ -251,7 +251,22 @@ func (sp *StreamProcessor) consumeStream(stream *es.Stream, closeWhenReady chan<
 }
 
 func (sp *StreamProcessor) subscribe(closeWhenReady chan<- struct{}) {
-	req, _ := http.NewRequest("GET", endpoints.AddPath(sp.streamURI, endpoints.StreamingRequestPath), nil)
+	req, reqErr := http.NewRequest("GET", endpoints.AddPath(sp.streamURI, endpoints.StreamingRequestPath), nil)
+	if reqErr != nil {
+		sp.loggers.Errorf(
+			"Unable to create a stream request; this is not a network problem, most likely a bad base URI: %s",
+			reqErr,
+		)
+		sp.dataSourceUpdates.UpdateStatus(interfaces.DataSourceStateOff, interfaces.DataSourceErrorInfo{
+			Kind:    interfaces.DataSourceErrorKindUnknown,
+			Message: reqErr.Error(),
+			Time:    time.Now(),
+		})
+		sp.logConnectionResult(false)
+		close(closeWhenReady)
+		return
+	}
+
 	for k, vv := range sp.headers {
 		req.Header[k] = vv
 	}

internal/datasource/streaming_data_source_test.go

@@ -565,3 +565,26 @@ func TestStreamProcessorRestartsStreamIfStoreNeedsRefresh(t *testing.T) {
 		})
 	})
 }
+
+func TestMalformedStreamBaseURI(t *testing.T) {
+	mockLog := ldlogtest.NewMockLog()
+	defer mockLog.DumpIfTestFailed(t)
+	clientContext := internal.NewClientContextImpl(
+		interfaces.BasicConfiguration{SDKKey: testSDKKey},
+		sharedtest.TestHTTPConfig(),
+		sharedtest.TestLoggingConfigWithLoggers(mockLog.Loggers),
+	)
+	withMockDataSourceUpdates(func(updates *sharedtest.MockDataSourceUpdates) {
+		sp := NewStreamProcessor(clientContext, updates, ":/", briefDelay)
+		defer sp.Close()
+
+		closeWhenReady := make(chan struct{})
+		sp.Start(closeWhenReady)
+
+		status := updates.RequireStatusOf(t, interfaces.DataSourceStateOff)
+		assert.Equal(t, interfaces.DataSourceErrorKindUnknown, status.LastError.Kind)
+		<-closeWhenReady
+
+		mockLog.AssertMessageMatch(t, true, ldlog.Error, "Unable to create a stream request")
+	})
+}

testhelpers/storetest/big_segment_store_test_suite.go

@@ -82,8 +82,11 @@ func (s *BigSegmentStoreTestSuite) runMetadataTests(t testbox.TestingT) {
 
 	t.Run("no value", func(t testbox.TestingT) {
 		s.withStoreAndEmptyData(t, func(store interfaces.BigSegmentStore) {
-			_, err := store.GetMetadata()
-			require.Error(t, err)
+			meta, err := store.GetMetadata()
+			// The Big Segment store should not return a database error in this case; it should return
+			// a result with an unset (zero) LastUpToDate, meaning "the store has not been updated ever".
+			assert.Equal(t, ldtime.UnixMillisecondTime(0), meta.LastUpToDate)
+			assert.NoError(t, err)
 		})
 	})
 }

testhelpers/storetest/big_segment_store_test_suite_test.go

@@ -48,7 +48,7 @@ func (s *mockSegmentStore) GetMetadata() (interfaces.BigSegmentStoreMetadata, er
 		return s.owner.overrideGetMetadata(s)
 	}
 	if s.metadata == nil {
-		return interfaces.BigSegmentStoreMetadata{}, errors.New("not found")
+		return interfaces.BigSegmentStoreMetadata{}, nil
 	}
 	return *s.metadata, nil
 }

testservice/go.mod

@@ -4,7 +4,7 @@ go 1.14
 
 require (
 	github.com/gorilla/mux v1.8.0
-	gopkg.in/launchdarkly/go-sdk-common.v2 v2.4.0
+	gopkg.in/launchdarkly/go-sdk-common.v2 v2.5.1
 	gopkg.in/launchdarkly/go-server-sdk.v5 v5.4.0
 )
 

testservice/go.sum

@@ -57,8 +57,9 @@ gopkg.in/launchdarkly/go-jsonstream.v1 v1.0.0/go.mod h1:YefdBjfITIP8D9BJLVbssFct
 gopkg.in/launchdarkly/go-jsonstream.v1 v1.0.1 h1:aZHvMDAS+M6/0sRMkDBQ8MyLGsTQrNgN5evu5e8UYpQ=
 gopkg.in/launchdarkly/go-jsonstream.v1 v1.0.1/go.mod h1:YefdBjfITIP8D9BJLVbssFctHkJnQXhv+TiRdTV0Jr4=
 gopkg.in/launchdarkly/go-sdk-common.v2 v2.2.1/go.mod h1:Fht0iTasUXh2xiDA8IJSmlSGbyQ1GNpmt97lXYz6+p8=
-gopkg.in/launchdarkly/go-sdk-common.v2 v2.4.0 h1:uA7it+cSIDIF4AhLoaLvQ5h9TxvSSVmn/CsJiAqrm4E=
 gopkg.in/launchdarkly/go-sdk-common.v2 v2.4.0/go.mod h1:P2+C6CHteys+lEDd6298QszCsMhjdYrfzBd6dg//CHA=
+gopkg.in/launchdarkly/go-sdk-common.v2 v2.5.1 h1:RqucG3hCU/GAupuEyVXXPf0Hz3F4InyhiFR2sfUbgBs=
+gopkg.in/launchdarkly/go-sdk-common.v2 v2.5.1/go.mod h1:P2+C6CHteys+lEDd6298QszCsMhjdYrfzBd6dg//CHA=
 gopkg.in/launchdarkly/go-sdk-events.v1 v1.1.1 h1:LfbZsHTPwjzhDbJ/IjYs0oc8rWcbyJM7nN+Ce4ZdUVM=
 gopkg.in/launchdarkly/go-sdk-events.v1 v1.1.1/go.mod h1:UETsxDtKpoDGUrwliXl1L7OG68zjOO0aagDI8OnvDRw=
 gopkg.in/launchdarkly/go-server-sdk-evaluation.v1 v1.5.0 h1:gA0F8n0sJ0K6LOLuyC28+O4garjdaU2T3m5mk1Fki8g=

testservice/sdk_client_entity.go

@@ -89,6 +89,9 @@ func (c *SDKClientEntity) DoCommand(params servicedef.CommandParams) (interface{
 	case servicedef.CommandGetBigSegmentStoreStatus:
 		bigSegmentsStatus := c.sdk.GetBigSegmentStoreStatusProvider().GetStatus()
 		return servicedef.BigSegmentStoreStatusResponse(bigSegmentsStatus), nil
+	case servicedef.CommandSecureModeHash:
+		hash := c.sdk.SecureModeHash(params.SecureModeHash.User)
+		return servicedef.SecureModeHashResponse{Result: hash}, nil
 	default:
 		return nil, BadRequestError{Message: fmt.Sprintf("unknown command %q", params.Command)}
 	}

testservice/service.go

@@ -29,6 +29,7 @@ var capabilities = []string{
 	servicedef.CapabilityAllFlagsDetailsOnlyForTrackedFlags,
 	servicedef.CapabilityAllFlagsWithReasons,
 	servicedef.CapabilityBigSegments,
+	servicedef.CapabilitySecureModeHash,
 	servicedef.CapabilityServerSidePolling,
 	servicedef.CapabilityServiceEndpoints,
 	servicedef.CapabilityTags,

testservice/servicedef/command_params.go

@@ -14,6 +14,7 @@ const (
 	CommandAliasEvent               = "aliasEvent"
 	CommandFlushEvents              = "flushEvents"
 	CommandGetBigSegmentStoreStatus = "getBigSegmentStoreStatus"
+	CommandSecureModeHash           = "secureModeHash"
 )
 
 type ValueType string
@@ -27,12 +28,13 @@ const (
 )
 
 type CommandParams struct {
-	Command       string                  `json:"command"`
-	Evaluate      *EvaluateFlagParams     `json:"evaluate,omitempty"`
-	EvaluateAll   *EvaluateAllFlagsParams `json:"evaluateAll,omitempty"`
-	CustomEvent   *CustomEventParams      `json:"customEvent,omitempty"`
-	IdentifyEvent *IdentifyEventParams    `json:"identifyEvent,omitempty"`
-	AliasEvent    *AliasEventParams       `json:"aliasEvent,omitempty"`
+	Command        string                  `json:"command"`
+	Evaluate       *EvaluateFlagParams     `json:"evaluate,omitempty"`
+	EvaluateAll    *EvaluateAllFlagsParams `json:"evaluateAll,omitempty"`
+	CustomEvent    *CustomEventParams      `json:"customEvent,omitempty"`
+	IdentifyEvent  *IdentifyEventParams    `json:"identifyEvent,omitempty"`
+	AliasEvent     *AliasEventParams       `json:"aliasEvent,omitempty"`
+	SecureModeHash *SecureModeHashParams   `json:"secureModeHash,omitempty"`
 }
 
 type EvaluateFlagParams struct {
@@ -81,3 +83,11 @@ type BigSegmentStoreStatusResponse struct {
 	Available bool `json:"available"`
 	Stale     bool `json:"stale"`
 }
+
+type SecureModeHashParams struct {
+	User lduser.User `json:"user"`
+}
+
+type SecureModeHashResponse struct {
+	Result string `json:"result"`
+}