Pick a single search head when search head cluster is detected #150
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,7 +4,7 @@ import * as vscode from 'vscode'; | |
| import { SplunkMessage } from './utils/messages'; | ||
| import { getModuleStatements } from './utils/parsing'; | ||
|
|
||
| export function getClient(): any { | ||
| const config = vscode.workspace.getConfiguration(); | ||
| const restUrl = config.get<string>('splunk.commands.splunkRestUrl'); | ||
| const token = config.get<string>('splunk.commands.token'); | ||
|
|
@@ -19,42 +19,16 @@ export function getClient() { | |
| host: host, | ||
| port: port, | ||
| sessionKey: token, | ||
| authorization: 'Bearer', | ||
| }); | ||
| service._originalURL = restUrl; | ||
|
|
||
| return service; | ||
| } | ||
|
|
||
| export function createSearchJob(jobs, query, options): Promise<any> { | ||
| let request = jobs.create(query, options); | ||
| return request; | ||
| } | ||
|
|
||
| /** | ||
|
|
@@ -69,6 +43,61 @@ function makeHeaders(service: any): object { | |
| }; | ||
| } | ||
|
|
||
| /** | ||
| * Check to see if the SDK client is part of a search head cluster, if so return a new | ||
| * client pointing to an individual search head member, such that any search ids created | ||
| * will be immediately available for results rather than waiting for artifact replication | ||
| * across the search head cluster. | ||
| * @param service Instance of the Javascript SDK Service | ||
| * | ||
| * @returns Promise<void> | ||
| */ | ||
| export function getSearchHeadClusterMemberClient(service: any): Promise<any> { | ||
|
There was a problem hiding this comment. nit: should the |
||
| const shcUrl = `${service.prefix}/services/shcluster/member/members?output_mode=json`; | ||
| console.log(`Attempting to determine SHC info if present using ${shcUrl}`); | ||
| return needle( | ||
| "GET", | ||
| shcUrl, | ||
| { | ||
| 'headers': makeHeaders(service), | ||
| 'followAllRedirects': true, | ||
| 'timeout': 0, | ||
| 'strictSSL': false, | ||
| 'rejectUnauthorized' : false, | ||
| }) | ||
| .then((response) => { | ||
| console.log(`Response from shcUrl status code: ${response.statusCode}`); | ||
| console.log(`Response from shcUrl body: \n'${JSON.stringify(response.body)}'`); | ||
| const data = response.body; | ||
| if (response.statusCode >= 400 || | ||
| !Object.prototype.isPrototypeOf(data) | ||
| || data.entry === undefined | ||
| || !Array.isArray(data.entry) | ||
| || data.entry.length === 0 | ||
| || data.entry[0].content === undefined | ||
| || data.entry[0].content.mgmt_uri === undefined | ||
| ) { | ||
| console.warn("Unsuccessful response from /services/shcluster/member/members endpoint encountered, reverting to original service client.") | ||
| return service; | ||
| } | ||
| // This is in the expected successful response format | ||
| vscode.window.showInformationMessage(`Discovered search head cluster members. Attempting to communicate directly with SH ${data.entry[0].content.mgmt_uri}`); | ||
| const url = new URL(data.entry[0].content.mgmt_uri); | ||
| const scheme = url.protocol.replace(':', ''); | ||
| const port = url.port || (scheme === 'https' ? '443' : '80'); | ||
| const host = url.hostname; | ||
| const newService = new splunk.Service({ | ||
| scheme: scheme, | ||
| host: host, | ||
| port: port, | ||
| sessionKey: service.sessionKey, | ||
| authorization: 'Bearer', | ||
| }); | ||
| newService._originalURL = service._originalURL; | ||
|
There was a problem hiding this comment. By keeping the original URL around we can decide whether the user has changed this value or not, if a new value was entered by the user in their settings then that will trigger creation of a new client and re-determining whether that is attached to a search head cluster. |
||
| return newService; | ||
| }); | ||
| } | ||
|
|
||
| /** | ||
| * Update a module by calling the PUT /services/spl2/modules/<namespace>.<moduleName> | ||
| * @param service Instance of the Javascript SDK Service | ||
|
|
@@ -77,7 +106,7 @@ function makeHeaders(service: any): object { | |
| * @param module Full contents of the module to update with | ||
| * @returns Promise<void> (or throw Error containing data.messages[]) | ||
| */ | ||
| export function updateSpl2Module(service: any, moduleName: string, namespace: string, module: string): Promise<void> { | ||
| // The Splunk SDK for Javascript doesn't currently support the spl2/modules endpoints | ||
| // nor does it support sending requests in JSON format (only receiving responses), so | ||
| // for now use the underlying needle library that the SDK uses for requests/responses | ||
|
|
@@ -131,9 +160,9 @@ export function updateSpl2Module(service: any, moduleName: string, namespace: st | |
| * @param namespace Namespace _within_ the apps.<app> to run, this will be used directly in the body of the request | ||
| * @param earliest Earliest time to be included in the body of the request | ||
| * @param latest Latest time to be included in the body of the request | ||
| * @returns A Promise containing the job information including sid created (or throw an Error containing data.messages[]) | ||
| */ | ||
| export function dispatchSpl2Module(service: any, spl2Module: string, app: string, namespace: string, earliest: string, latest: string): Promise<any> { | ||
| // For now we're using /services/<app> which doesn't respect relative namespaces, | ||
| // so for now hardcode this to '' but if/when /servicesNS/<app> | ||
| namespace = ''; | ||
|
|
@@ -194,6 +223,7 @@ export function dispatchSpl2Module(service: any, spl2Module: string, app: string | |
| .then((response) => { | ||
| console.log(`Response status code: ${response.statusCode}`); | ||
| console.log(`Response body: \n'${JSON.stringify(response.body)}'`); | ||
| console.log(`Response headers: \n'${JSON.stringify(response.headers)}'`); | ||
| const data = response.body; | ||
| if (response.statusCode >= 400 || !Array.prototype.isPrototypeOf(data) || data.length < 1) { | ||
| handleErrorPayloads(data, response.statusCode); | ||
|
|
@@ -267,72 +297,33 @@ function handleErrorPayloads(data: any, statusCode: number) { | |
| }); | ||
| } | ||
|
|
||
| export function getSearchJobBySid(service, sid): Promise<any> { | ||
| let request = service.getJob(sid); | ||
|
There was a problem hiding this comment. All of these refactors were necessary after upgrading the Javascript SDK to version 2+ see: https://github.com/splunk/splunk-sdk-javascript?tab=readme-ov-file#migrate-from-callbacksv1x-to-promiseasync-awaitv2x There was a problem hiding this comment. In the callback version of the helper code here and in other places there was some error handling that I'm not seeing in the promise version, is this no longer necessary? I.e I would have expected to see some try/catch blocks. There was a problem hiding this comment. so the resolve/reject handling that I removed is just part of the Promise formation and that's done in the SDK now. For these functions the expectation is that error handling is happening by the caller, which you can see happening here for the SPL2 case for example: vscode-extension-splunk/out/notebooks/spl2/controller.ts Lines 37 to 59 in 4bda67d |
||
| return request; | ||
| } | ||
|
|
||
|
|
||
| export function getSearchJob(job): Promise<any> { | ||
| let request = job.fetch(); | ||
| return request; | ||
| } | ||
|
|
||
| export function getJobSearchLog(job): Promise<any> { | ||
| let request = job.searchlog(); | ||
| return request; | ||
| } | ||
|
|
||
| export function getSearchJobResults(job): Promise<any> { | ||
| let request = job.get("results", {"output_mode": "json_cols"}); | ||
| return request; | ||
| } | ||
|
|
||
| export function cancelSearchJob(job): Promise<any> { | ||
| let request = job.cancel(); | ||
| return request; | ||
| } | ||
|
|
||
| export function wait(ms = 1000): Promise<void> { | ||
| return new Promise(resolve => { | ||
| setTimeout(resolve, ms); | ||
| }); | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this only be called if
this._serviceis null?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah it would be
undefinedbut let me fix this to be a bit more specificThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done