Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.MD
Original file line number Diff line number Diff line change
Expand Up @@ -763,6 +763,7 @@ A reporter is a class that is able to report on the progress of the test run. In
- `ProgressReporter` - prints the result of each scenario and step to the console - colours the output.
- `TestRunSummaryReporter` - prints the results and duration of the test run once the run has completed - colours the output.
- `JsonReporter` - creates a JSON file with the results of the test run which can then be used by 'https://www.npmjs.com/package/cucumber-html-reporter.' to create a HTML report. You can pass in the file path of the json file to be created.
- `SlackReporter` - notifies a Slack channel for every failed step and every successful feature. This requires a bot installed to your workspace with the files:write scope (to upload screenshots of failed steps) and the chat.write permission (to publish test results).

You can create your own custom reporter by inheriting from the base `Reporter` class and overriding the one or many of the methods to direct the output message. The `Reporter` defines the following methods that can be overridden. All methods must return a `Future<void>` and can be async.

Expand Down
1 change: 1 addition & 0 deletions lib/gherkin.dart
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ export 'src/reporters/message_level.dart';
export 'src/reporters/messages.dart';
export 'src/reporters/stdout_reporter.dart';
export 'src/reporters/progress_reporter.dart';
export 'src/reporters/slack_reporter.dart';
export 'src/reporters/test_run_summary_reporter.dart';
export 'src/reporters/json/json_reporter.dart';

Expand Down
255 changes: 255 additions & 0 deletions lib/src/reporters/slack_reporter.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,255 @@
import 'dart:async';
import 'dart:io';
import 'dart:convert';

import 'package:gherkin/gherkin.dart';
import 'package:meta/meta.dart';

import 'package:http/http.dart' as http;
import 'package:http_parser/http_parser.dart' show MediaType;

/// Upload messages and logs as the delivery layer for the [SlackReporter].
class SlackMessenger {
/// After [creating a Slack Bot](https://api.slack.com/apps) with
/// the proper scopes (chat.write, files:write), an Oauth token will be
/// generated after installing for your Slack team (https://api.slack.com/apps/xxx/install-on-team)
///
/// If the images aren't uploading, test your token on Slack: https://api.slack.com/methods/files.upload/test
/// Often, the solution is to invite the bot to the proper Slack channel.
final String slackOauthToken;

/// Easiest way to find this: right click the channel name in Slack>"Open Link" .
/// Use the alphanumeric ending of the URL.
final String slackChannelId;

/// The parent message ID. This will nest the test run to reduce channel noise.
String threadTs;

SlackMessenger({
@required this.slackOauthToken,
@required this.slackChannelId,
});

/// Format a Slack text Block
Map<String, dynamic> buildTextSection(String txt) => {
'type': 'section',
'text': {'type': 'mrkdwn', 'text': txt}
};

/// Slack Block divider
Map<String, dynamic> get divider => {'type': 'divider'};

Future<void> message(msg, level) async {
if (level == MessageLevel.error) await sendText(msg);
}

Future<http.Response> notifySlack(List<Map<String, dynamic>> payload) => http
.post(
'https://slack.com/api/chat.postMessage',
headers: {
'Content-Type': 'application/json',
'Authorization': 'Bearer $slackOauthToken',
},
body: jsonEncode({
'channel': slackChannelId,
'blocks': payload,
if (threadTs != null) 'thread_ts': threadTs,
}),
)
.catchError(print);

/// Send normal text. Slack-supported Markdown is available.
Future<http.Response> sendText(String msg) {
final payload = [buildTextSection(msg)];
return notifySlack(payload);
}

/// Initial message will set [threadTs] so that subsequent
/// messages reply under one thread when [threadMessages] is true.
Future<http.Response> start(String startMessage,
{bool threadMessages = true}) async {
threadTs = null;
final resp = await sendText(startMessage);
if (threadMessages) threadTs = jsonDecode(resp.body)['ts'];
return resp;
}

/// Upload a file to Slack.
/// [contentType] inherits from the default [http.MultipartFile.fromBytes] argument.
Future<void> uploadFile(List<int> bytes, String title,
{MediaType contentType}) {
try {
if (slackOauthToken != null) {
// pulled from https://api.slack.com/methods/files.upload/test
final endpoint = _slackEndpoint('files.upload', {
'channels': slackChannelId,
if (threadTs != null) 'thread_ts': threadTs,
'title': title,
});
final request = http.MultipartRequest('POST', Uri.parse(endpoint));
final file = http.MultipartFile.fromBytes('file', bytes,
filename: '$title.png', contentType: contentType);
request.files.add(file);
return request.send();
}
} catch (e, st) {
print('Failed to upload file\n$e\n$st');
}

return null;
}

String _slackEndpoint(String method, Map<String, String> queryParameters) {
final baseUrl = 'https://slack.com/api/$method?token=$slackOauthToken';
return queryParameters.entries.fold(baseUrl, (acc, entry) {
final encoded = Uri.encodeComponent(entry.value);
return '$acc&${entry.key}=$encoded';
});
}
}

/// Notify a Slack channel for every failed step and every successful feature.
/// This requires a bot installed to your workspace with the files:write scope
/// (to upload screenshots of failed steps) and the chat.write permission
/// (to publish test results).
///
/// Image attachments will be uploaded for failed steps if generated by the step or
/// some other hook. For example, a screenshot hook:
/// ```dart
/// class AttachScreenshotAfterStepHook extends Hook {
/// @override
/// Future<void> onAfterStep(World world, String step, StepResult stepResult) async {
/// try {
/// final bytes = await (world as FlutterWorld).driver.screenshot();
/// world.attach(base64Encode(bytes), 'image/png', step);
/// } catch (e, st) {
/// print('Failed to take screenshot\n$e\n$st');
/// }
/// }
/// }
/// ```
///
/// Refer to [SlackMessenger] for steps to setup and configure your bot.
class SlackReporter extends Reporter {
/// The number of scenario failures before the test suite will terminate
/// and cease reporting to Slack. Defaults to `15`.
final int maximumToleratedFailures;

final SlackMessenger slackMessenger;

/// Reported as the initial message (or parent thread when [threadResults] is `true`)
/// of the test run. Defaults to `Starting tests`.
final String startLabel;

/// Whether the test run should be threaded under a single message. Defaults `true`.
final bool threadResults;

final features = <FinishedMessage>[];
final scenarios = <ScenarioFinishedMessage>[];
final scenariosInActiveFeature = <ScenarioFinishedMessage>[];
StepFinishedMessage firstFailedStepInActiveScenario;

SlackReporter(
this.slackMessenger, {
this.maximumToleratedFailures = 15,
this.threadResults = true,
this.startLabel = 'Starting tests',
});

@override
Future<void> onException(exception, stackTrace) {
final payload = [
slackMessenger.divider,
slackMessenger.buildTextSection(':bangbang: Exception :bangbang:'),
slackMessenger.buildTextSection(exception.toString()),
slackMessenger.buildTextSection('```$stackTrace```'),
slackMessenger.divider,
];

return slackMessenger.notifySlack(payload);
}

@override
Future<void> message(msg, level) async {
if (level == MessageLevel.error) await slackMessenger.sendText(msg);
}

@override
Future<void> onFeatureFinished(feature) async {
features.add(feature);
final allScenariosPassed = scenariosInActiveFeature.every((s) => s.passed);

if (allScenariosPassed) {
await slackMessenger.sendText(
':white_check_mark: ${feature.name} (${scenariosInActiveFeature.length}/${scenariosInActiveFeature.length} passed)');
} else {
final passedScenarios = scenariosInActiveFeature.where((s) => s.passed);
await slackMessenger.sendText(
':warning: ${feature.name} (${passedScenarios.length}/${scenariosInActiveFeature.length} passed)');
}

scenariosInActiveFeature.clear();
}

@override
Future<void> onScenarioFinished(scenario) async {
scenarios.add(scenario);
scenariosInActiveFeature.add(scenario);
if (!scenario.passed) {
final payload = [
slackMessenger.buildTextSection(':x: ${scenario.name}'),
slackMessenger.buildTextSection(
'Failed at step: ${firstFailedStepInActiveScenario.name}'),
];
await slackMessenger
.notifySlack(payload)
.then((_) => firstFailedStepInActiveScenario = null);
}

if (scenarios.where((s) => !s.passed).length > maximumToleratedFailures) {
await slackMessenger.sendText(
':x: :x: :x: :x: Aborting: too many failures :x: :x: :x: :x:');
exit(1);
}
}

@override
Future<void> onStepFinished(step) async {
if (step.result.result != StepExecutionResult.pass &&
firstFailedStepInActiveScenario == null) {
firstFailedStepInActiveScenario = step;
for (var attachment in step.attachments) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jonsamwell I added this here to address your feedback. As this repo doesn't access FlutterWorld, it didn't make sense to include the hook that generates the screenshot. I documented it for the implementor's convenience. Do you think that example is sufficient or should it be more visible/cross-documented?

if (attachment.mimeType == 'image/png') {
final bytes = base64Decode(attachment.data);
await slackMessenger.uploadFile(bytes, step.name);
}
}
}
}

@override
Future<void> onTestRunStarted() async =>
slackMessenger.start(startLabel, threadMessages: threadResults);

@override
Future<void> onTestRunFinished() async {
final successfulNames =
scenarios.where((s) => s.passed).map((s) => '* ${s.name}');
final failedNames =
scenarios.where((s) => !s.passed).map((s) => '* ${s.name}');

final payload = [
slackMessenger.buildTextSection('Testing Complete'),
slackMessenger.divider,
slackMessenger.buildTextSection(
':white_check_mark: ${successfulNames.length} / ${scenarios.length} Successful Tests:'),
slackMessenger.buildTextSection(successfulNames.join('\n')),
slackMessenger.divider,
slackMessenger.buildTextSection(
':x: ${failedNames.length} / ${scenarios.length} Failed Tests:'),
slackMessenger.buildTextSection(failedNames.join('\n')),
];

return slackMessenger.notifySlack(payload);
}
}