security: make allow_net restrict file:// refs in JSON schemas - #9044
security: make allow_net restrict file:// refs in JSON schemas#9044HarnageaGabriel wants to merge 4 commits into
Conversation
jsonReferenceLoader.LoadJSON resolved file:// $refs by calling loadFromFile unconditionally, before the isAllowed allow-list check. isAllowed only gates the HTTP branch, so allow_net (wired in via newPatternValidatingSchemaLoader for json.match_schema and json.verify_schema) had no effect on file:// refs at all -- even allow_net: [] left local file reads on the machine running OPA completely open. Since the schema operand to those builtins can come from input, this allowed input-driven arbitrary local file reads. Deny file:// refs whenever allow_net is set (including the empty-list case), matching how allow_net already treats every other remote resource: deny by default unless explicitly permitted. When allow_net is unset, file refs keep loading as before. Fixes open-policy-agent#9000 Signed-off-by: Gabriel Harnagea <gabriel.harnagea06@gmail.com>
| } | ||
|
|
||
| func TestAllowNetRestrictsFileReferences(t *testing.T) { | ||
| filename := filepath.Join(t.TempDir(), "schema.json") |
There was a problem hiding this comment.
might be able to use https://github.com/open-policy-agent/opa/blob/main/v1/util/test/tempfs.go#L20
|
|
||
| ### Fixes | ||
|
|
||
| - Security: Make `allow_net` restrict `file://` references in JSON schemas used by |
There was a problem hiding this comment.
Generally this is just updated during the release process.
| allowNet []string | ||
| wantDenied bool | ||
| }{ | ||
| {note: "nil list permits file references", allowNet: nil}, |
There was a problem hiding this comment.
Setting allow_net: [], to block file access feels incorrect here.
I think that we ought to default to not allowing file:// URIs in schemas. We might want to add another capability to allow this, that defaults to never allowing local file access.
AllowFileSystemPaths: default [] is one idea.
Revert the CHANGELOG.md entry -- per @charlieegan3, that file is updated during the release process, not by individual PRs. Switch the file-reference test to the existing test.TempDirOf helper from v1/util/test instead of manually building the temp file, per @charlieegan3's pointer to that helper. Note: charlieegan3 also suggested decoupling the file:// deny from allow_net entirely (a dedicated capability defaulting to deny, independent of whether allow_net is set). Tried that as a follow-up and confirmed it regresses TestFragmentLoader and TestFileWithSpace, which load local schema files directly (not via an untrusted ) and never set allow_net -- so an unconditional deny breaks legitimate trusted-caller file loading, not just the vulnerable path. Left the allow_net-gated behavior in place pending a properly scoped follow-up that only denies for the json.verify_schema/json.match_schema builtin path specifically. Signed-off-by: Gabriel Harnagea <gabriel.harnagea06@gmail.com>
Revert the CHANGELOG.md entry -- per @charlieegan3, that file is updated during the release process, not by individual PRs. Switch the file-reference test to the existing test.TempDirOf helper from v1/util/test instead of manually building the temp file, per @charlieegan3's pointer to that helper. Note: charlieegan3 also suggested decoupling the file:// deny from allow_net entirely (a dedicated capability defaulting to deny, independent of whether allow_net is set). Tried that as a follow-up and confirmed it regresses TestFragmentLoader and TestFileWithSpace, which load local schema files directly (not via an untrusted ) and never set allow_net -- so an unconditional deny breaks legitimate trusted-caller file loading, not just the vulnerable path. Left the allow_net-gated behavior in place pending a properly scoped follow-up that only denies for the json.verify_schema/json.match_schema builtin path specifically.
|
Thanks for the review. Pushed two of the three fixes: reverted the CHANGELOG.md entry, and switched the test to On the third one (decoupling file:// deny from |
Fair enough, but I think if we want to actually address #9000 that is what's needed. I do not think that reusing allow_net is appropriate here. We need to have two different paths for schema loading for both |
allow_net previously doubled as the gate for file:// $ref access, which conflated remote-fetch policy with local-file policy and forced trusted callers (compile-time schema annotations) and untrusted callers (the json.verify_schema/json.match_schema builtins, where schemas may come from input) to share one on/off switch. Add a dedicated DenyFileScheme flag to gojsonschema.SchemaLoader, independent of AllowNet. newPatternValidatingSchemaLoader (backing the builtins) now sets it unconditionally, so file:// refs are denied regardless of allow_net's value. v1/ast/compile.go's compile-time loader leaves it unset, preserving existing trusted file-loading behavior. AllowNet goes back to governing only remote HTTP fetches. Signed-off-by: Gabriel Harnagea <gabriel.harnagea06@gmail.com>
|
Pushed
Tests:
Full runs: |
| @@ -55,6 +55,10 @@ type remoteRefLimits struct { | |||
There was a problem hiding this comment.
withRemoteRefLimits and remoteRefLimits are private, so we might want to consider a rename to something more generic like loaderLimits or loadPolicy since file system access is not actually "remote".
|
|
||
| // A file:// $ref is denied whenever DenyFileScheme is set, independent | ||
| // of allowNet. allowNet governs only remote HTTP fetches. | ||
| denyFileScheme bool |
There was a problem hiding this comment.
I might be inclined to call this allowFilesystem to match the polarity of allowNet, just make it default false.
|
Hi there, two small comments. Please also make sure that DCO is passing, this needs to be your email in the commit messages. |
d7349b1 to
136fad1
Compare
Per @charlieegan3's naming suggestions on PR open-policy-agent#9044: - remoteRefLimits/withRemoteRefLimits -> loaderLimits/withLoaderLimits, since filesystem access was never actually "remote". - DenyFileScheme -> AllowFilesystem (default false), matching AllowNet's polarity. Trusted callers (compile.go's compileSchema, schema.go's NewSchema) now explicitly set AllowFilesystem: true; the builtin-facing path (topdown/jsonschema.go) leaves it at its deny-by-default zero value instead of explicitly setting a deny flag. Signed-off-by: Gabriel Harnagea <gabriel.harnagea06@gmail.com>
|
Fixed DCO (two commits were missing sign-off) and applied both naming suggestions: |
| }{ | ||
| {note: "nil list", allowNet: nil}, | ||
| {note: "empty list", allowNet: []string{}}, | ||
| {note: "populated list", allowNet: []string{"example.com"}}, |
There was a problem hiding this comment.
I'd not really expect to see these tests based on allowNet now we are using a new limit setting.
|
Hi thanks for coming back to this, can you have a look at the above comment and rebase this? I think we just need to make sure the tests are updated too and testing the correct limit option, not allowNet. |
| // Local file access is allowed only when allowFilesystem is set, independent | ||
| // of allowNet. allowNet governs only remote HTTP fetches. |
There was a problem hiding this comment.
| // Local file access is allowed only when allowFilesystem is set, independent | |
| // of allowNet. allowNet governs only remote HTTP fetches. | |
| // Local file access is allowed only when allowFilesystem is set. |
Allownet is 'unrelated' as far as the field here is listed.
| capabilities: &ast.Capabilities{}, | ||
| }, | ||
| { | ||
| note: "empty allow_net", |
There was a problem hiding this comment.
Again, this is strange as it's allow_net, but then a file schema ref in the JSON above.
Fixes #9000
Root cause
jsonReferenceLoader.LoadJSON()ininternal/gojsonschema/jsonLoader.goresolvedfile://$refs by callingloadFromFile()unconditionally wheneverreference.HasFileSchemeis true — before theisAllowed()allow-list check is ever consulted:isAllowed()checksref.Hostname()againstlimits.allowNet, which is only meaningful for the HTTP branch.allow_net(wired in forjson.match_schema/json.verify_schemavianewPatternValidatingSchemaLoaderinv1/topdown/jsonschema.go) therefore never restrictedfile://refs at all —allow_net: []blocked every remote host but left local file reads on the machine running OPA completely open. Since the schema operand to these builtins can come frominput, this allowed input-driven arbitrary local file reads, defeating the sandboxingallow_netis meant to provide.Fix
Deny
file://refs wheneverallow_netis set (including the empty-list case,allow_net: []), consistent with howallow_netalready treats every other remote resource: deny by default unless explicitly permitted. Whenallow_netis unset (nil / unrestricted), file refs keep loading exactly as before — no behavior change for callers who don't configureallow_net.I considered instead extending the allow-list mechanism to permit specific local paths, but a hostname allow-list doesn't map cleanly onto filesystem paths, so I went with the more conservative deny-outright approach. Flagged this tradeoff on the issue before starting — happy to adjust if maintainers prefer a different shape (e.g. a dedicated capability for local file refs).
Testing
go build ./...passesgo test ./internal/gojsonschema/... ./v1/topdown/...passes (2 pre-existing, unrelatedTestCertSelectionLogicfailures reproduce identically on unmodifiedmainon this machine — Windows system-cert environment issue, not touched by this change)TestAllowNetRestrictsFileReferencescovering:file://ref denied whenallow_net: [],file://ref still loads whenallow_netunsetallow_netenforcement tests unmodified and still passChangelog
Added an entry under
Unreleased/FixesinCHANGELOG.md.