-
Notifications
You must be signed in to change notification settings - Fork 51
add Slack reporter (from jonsamwell/flutter_gherkin#71) #13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
tshedor
wants to merge
1
commit into
jonsamwell:master
Choose a base branch
from
tshedor:add-slack-reporter
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,255 @@ | ||
| import 'dart:async'; | ||
| import 'dart:io'; | ||
| import 'dart:convert'; | ||
|
|
||
| import 'package:gherkin/gherkin.dart'; | ||
| import 'package:meta/meta.dart'; | ||
|
|
||
| import 'package:http/http.dart' as http; | ||
| import 'package:http_parser/http_parser.dart' show MediaType; | ||
|
|
||
| /// Upload messages and logs as the delivery layer for the [SlackReporter]. | ||
| class SlackMessenger { | ||
| /// After [creating a Slack Bot](https://api.slack.com/apps) with | ||
| /// the proper scopes (chat.write, files:write), an Oauth token will be | ||
| /// generated after installing for your Slack team (https://api.slack.com/apps/xxx/install-on-team) | ||
| /// | ||
| /// If the images aren't uploading, test your token on Slack: https://api.slack.com/methods/files.upload/test | ||
| /// Often, the solution is to invite the bot to the proper Slack channel. | ||
| final String slackOauthToken; | ||
|
|
||
| /// Easiest way to find this: right click the channel name in Slack>"Open Link" . | ||
| /// Use the alphanumeric ending of the URL. | ||
| final String slackChannelId; | ||
|
|
||
| /// The parent message ID. This will nest the test run to reduce channel noise. | ||
| String threadTs; | ||
|
|
||
| SlackMessenger({ | ||
| @required this.slackOauthToken, | ||
| @required this.slackChannelId, | ||
| }); | ||
|
|
||
| /// Format a Slack text Block | ||
| Map<String, dynamic> buildTextSection(String txt) => { | ||
| 'type': 'section', | ||
| 'text': {'type': 'mrkdwn', 'text': txt} | ||
| }; | ||
|
|
||
| /// Slack Block divider | ||
| Map<String, dynamic> get divider => {'type': 'divider'}; | ||
|
|
||
| Future<void> message(msg, level) async { | ||
| if (level == MessageLevel.error) await sendText(msg); | ||
| } | ||
|
|
||
| Future<http.Response> notifySlack(List<Map<String, dynamic>> payload) => http | ||
| .post( | ||
| 'https://slack.com/api/chat.postMessage', | ||
| headers: { | ||
| 'Content-Type': 'application/json', | ||
| 'Authorization': 'Bearer $slackOauthToken', | ||
| }, | ||
| body: jsonEncode({ | ||
| 'channel': slackChannelId, | ||
| 'blocks': payload, | ||
| if (threadTs != null) 'thread_ts': threadTs, | ||
| }), | ||
| ) | ||
| .catchError(print); | ||
|
|
||
| /// Send normal text. Slack-supported Markdown is available. | ||
| Future<http.Response> sendText(String msg) { | ||
| final payload = [buildTextSection(msg)]; | ||
| return notifySlack(payload); | ||
| } | ||
|
|
||
| /// Initial message will set [threadTs] so that subsequent | ||
| /// messages reply under one thread when [threadMessages] is true. | ||
| Future<http.Response> start(String startMessage, | ||
| {bool threadMessages = true}) async { | ||
| threadTs = null; | ||
| final resp = await sendText(startMessage); | ||
| if (threadMessages) threadTs = jsonDecode(resp.body)['ts']; | ||
| return resp; | ||
| } | ||
|
|
||
| /// Upload a file to Slack. | ||
| /// [contentType] inherits from the default [http.MultipartFile.fromBytes] argument. | ||
| Future<void> uploadFile(List<int> bytes, String title, | ||
| {MediaType contentType}) { | ||
| try { | ||
| if (slackOauthToken != null) { | ||
| // pulled from https://api.slack.com/methods/files.upload/test | ||
| final endpoint = _slackEndpoint('files.upload', { | ||
| 'channels': slackChannelId, | ||
| if (threadTs != null) 'thread_ts': threadTs, | ||
| 'title': title, | ||
| }); | ||
| final request = http.MultipartRequest('POST', Uri.parse(endpoint)); | ||
| final file = http.MultipartFile.fromBytes('file', bytes, | ||
| filename: '$title.png', contentType: contentType); | ||
| request.files.add(file); | ||
| return request.send(); | ||
| } | ||
| } catch (e, st) { | ||
| print('Failed to upload file\n$e\n$st'); | ||
| } | ||
|
|
||
| return null; | ||
| } | ||
|
|
||
| String _slackEndpoint(String method, Map<String, String> queryParameters) { | ||
| final baseUrl = 'https://slack.com/api/$method?token=$slackOauthToken'; | ||
| return queryParameters.entries.fold(baseUrl, (acc, entry) { | ||
| final encoded = Uri.encodeComponent(entry.value); | ||
| return '$acc&${entry.key}=$encoded'; | ||
| }); | ||
| } | ||
| } | ||
|
|
||
| /// Notify a Slack channel for every failed step and every successful feature. | ||
| /// This requires a bot installed to your workspace with the files:write scope | ||
| /// (to upload screenshots of failed steps) and the chat.write permission | ||
| /// (to publish test results). | ||
| /// | ||
| /// Image attachments will be uploaded for failed steps if generated by the step or | ||
| /// some other hook. For example, a screenshot hook: | ||
| /// ```dart | ||
| /// class AttachScreenshotAfterStepHook extends Hook { | ||
| /// @override | ||
| /// Future<void> onAfterStep(World world, String step, StepResult stepResult) async { | ||
| /// try { | ||
| /// final bytes = await (world as FlutterWorld).driver.screenshot(); | ||
| /// world.attach(base64Encode(bytes), 'image/png', step); | ||
| /// } catch (e, st) { | ||
| /// print('Failed to take screenshot\n$e\n$st'); | ||
| /// } | ||
| /// } | ||
| /// } | ||
| /// ``` | ||
| /// | ||
| /// Refer to [SlackMessenger] for steps to setup and configure your bot. | ||
| class SlackReporter extends Reporter { | ||
| /// The number of scenario failures before the test suite will terminate | ||
| /// and cease reporting to Slack. Defaults to `15`. | ||
| final int maximumToleratedFailures; | ||
|
|
||
| final SlackMessenger slackMessenger; | ||
|
|
||
| /// Reported as the initial message (or parent thread when [threadResults] is `true`) | ||
| /// of the test run. Defaults to `Starting tests`. | ||
| final String startLabel; | ||
|
|
||
| /// Whether the test run should be threaded under a single message. Defaults `true`. | ||
| final bool threadResults; | ||
|
|
||
| final features = <FinishedMessage>[]; | ||
| final scenarios = <ScenarioFinishedMessage>[]; | ||
| final scenariosInActiveFeature = <ScenarioFinishedMessage>[]; | ||
| StepFinishedMessage firstFailedStepInActiveScenario; | ||
|
|
||
| SlackReporter( | ||
| this.slackMessenger, { | ||
| this.maximumToleratedFailures = 15, | ||
| this.threadResults = true, | ||
| this.startLabel = 'Starting tests', | ||
| }); | ||
|
|
||
| @override | ||
| Future<void> onException(exception, stackTrace) { | ||
| final payload = [ | ||
| slackMessenger.divider, | ||
| slackMessenger.buildTextSection(':bangbang: Exception :bangbang:'), | ||
| slackMessenger.buildTextSection(exception.toString()), | ||
| slackMessenger.buildTextSection('```$stackTrace```'), | ||
| slackMessenger.divider, | ||
| ]; | ||
|
|
||
| return slackMessenger.notifySlack(payload); | ||
| } | ||
|
|
||
| @override | ||
| Future<void> message(msg, level) async { | ||
| if (level == MessageLevel.error) await slackMessenger.sendText(msg); | ||
| } | ||
|
|
||
| @override | ||
| Future<void> onFeatureFinished(feature) async { | ||
| features.add(feature); | ||
| final allScenariosPassed = scenariosInActiveFeature.every((s) => s.passed); | ||
|
|
||
| if (allScenariosPassed) { | ||
| await slackMessenger.sendText( | ||
| ':white_check_mark: ${feature.name} (${scenariosInActiveFeature.length}/${scenariosInActiveFeature.length} passed)'); | ||
| } else { | ||
| final passedScenarios = scenariosInActiveFeature.where((s) => s.passed); | ||
| await slackMessenger.sendText( | ||
| ':warning: ${feature.name} (${passedScenarios.length}/${scenariosInActiveFeature.length} passed)'); | ||
| } | ||
|
|
||
| scenariosInActiveFeature.clear(); | ||
| } | ||
|
|
||
| @override | ||
| Future<void> onScenarioFinished(scenario) async { | ||
| scenarios.add(scenario); | ||
| scenariosInActiveFeature.add(scenario); | ||
| if (!scenario.passed) { | ||
| final payload = [ | ||
| slackMessenger.buildTextSection(':x: ${scenario.name}'), | ||
| slackMessenger.buildTextSection( | ||
| 'Failed at step: ${firstFailedStepInActiveScenario.name}'), | ||
| ]; | ||
| await slackMessenger | ||
| .notifySlack(payload) | ||
| .then((_) => firstFailedStepInActiveScenario = null); | ||
| } | ||
|
|
||
| if (scenarios.where((s) => !s.passed).length > maximumToleratedFailures) { | ||
| await slackMessenger.sendText( | ||
| ':x: :x: :x: :x: Aborting: too many failures :x: :x: :x: :x:'); | ||
| exit(1); | ||
| } | ||
| } | ||
|
|
||
| @override | ||
| Future<void> onStepFinished(step) async { | ||
| if (step.result.result != StepExecutionResult.pass && | ||
| firstFailedStepInActiveScenario == null) { | ||
| firstFailedStepInActiveScenario = step; | ||
| for (var attachment in step.attachments) { | ||
| if (attachment.mimeType == 'image/png') { | ||
| final bytes = base64Decode(attachment.data); | ||
| await slackMessenger.uploadFile(bytes, step.name); | ||
| } | ||
| } | ||
| } | ||
| } | ||
|
|
||
| @override | ||
| Future<void> onTestRunStarted() async => | ||
| slackMessenger.start(startLabel, threadMessages: threadResults); | ||
|
|
||
| @override | ||
| Future<void> onTestRunFinished() async { | ||
| final successfulNames = | ||
| scenarios.where((s) => s.passed).map((s) => '* ${s.name}'); | ||
| final failedNames = | ||
| scenarios.where((s) => !s.passed).map((s) => '* ${s.name}'); | ||
|
|
||
| final payload = [ | ||
| slackMessenger.buildTextSection('Testing Complete'), | ||
| slackMessenger.divider, | ||
| slackMessenger.buildTextSection( | ||
| ':white_check_mark: ${successfulNames.length} / ${scenarios.length} Successful Tests:'), | ||
| slackMessenger.buildTextSection(successfulNames.join('\n')), | ||
| slackMessenger.divider, | ||
| slackMessenger.buildTextSection( | ||
| ':x: ${failedNames.length} / ${scenarios.length} Failed Tests:'), | ||
| slackMessenger.buildTextSection(failedNames.join('\n')), | ||
| ]; | ||
|
|
||
| return slackMessenger.notifySlack(payload); | ||
| } | ||
| } | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jonsamwell I added this here to address your feedback. As this repo doesn't access
FlutterWorld, it didn't make sense to include the hook that generates the screenshot. I documented it for the implementor's convenience. Do you think that example is sufficient or should it be more visible/cross-documented?