Skip to content

Commit d3cb8cd

Browse files
scetronscetron
authored
Merge pull request #61 from networktocode-llc/update_readme
Update docs as per OSRB request
2 parents 7b3424d + 60c8fd8 commit d3cb8cd

File tree

2 files changed

+98
-60
lines changed

2 files changed

+98
-60
lines changed

‎README.md

Lines changed: 98 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,59 @@
11
# netcompare
22

3-
This library is meant to be a light-weight way to compare structured output from network devices `show` commands. `netcompare` is a python library targeted at intelligently deep diffing structured data objects of json type. In addition, `netcompare` can also provide some basic testing of key/values within a data structure.
3+
The `netcompare` is a light-weight library to examine structured data. `netcompare` provides an interface to intelligently compare json data objects as well as test for the presence (or absence) of keys, and to examine and compare their values.
44

5-
The library heavily relies on [jmespath](https://jmespath.org/) for traversing the json object and finding the value(s) to be evaluated. More on that later.
5+
The library heavily relies on [jmespath](https://jmespath.org/) for traversing the json object and finding the value(s) to be evaluated. More on that [here](#customized-jmespath).
66

7-
## Use Case
7+
## Usage
88

9-
`netcompare` enables an easy and direct way to see the outcome of network configuration or operational status change. The intended usage is to collect structured `show` command output before and after a change window. Prior to closing the change window, the results are compared to help determine if the change was successful as intended and if the network is in an acceptable state. The output can be stored with the change's documentation for easy reference and proof of completion.
9+
A `netcompare` Check accepts two objects as input: the reference object, and the comparison object. The reference object is used as the intended or accepted state and it's keys and values are compared against the comparison object.
10+
11+
`netcompare` does not collect the data for you, it simply works on data passed into it. This allows for maximum flexibility in collecting the data.
12+
13+
For instance, the reference state can be collected from the network directly using any method that returns structured data: ansible, napalm, nornir to name a few. You could also choose to generate the reference state from an SoT, such as [Nautobot](https://github.com/nautobot/nautobot/), and have a true intended state.
14+
15+
`netcompare` is perfectly suited to work with data gathered from network devices via show commands, Ansible playbooks, as well as in applications such as [Nautobot](https://github.com/nautobot/nautobot/), or [Netbox](https://github.com/netbox-community/netbox). `netcompare` is focused on being the 'plumbing' behind a full network automation validation solution.
16+
17+
### Testing data structures
18+
19+
Briefly, these tests, or `CheckTypes`, are provided to test the objects, to aide in determining the status of the data.
20+
21+
- `exact_match`: the keys and values much match, exactly, between the two objects
22+
- `tolerance`: the keys must match and the values can differ according to the 'tolerance' value provided
23+
- `parameter_match`: a reference key and value is provided and it's presence (or absence) is checked in the provided object
24+
- `regex`: a reference regex pattern is provided and it's presence (or absence) is checked for a match in the provided object
25+
- `operator`: similar to parameter match, but the reference includes several different possible operators: 'in', 'bool', 'string', and numerical comparison with 'int' and 'float' to check against
26+
27+
`CheckTypes` are explained in more detail in the [CheckTypes Explained section](#check-types-explained).
28+
29+
30+
## Workflow
31+
32+
| ![netcompare workflow](./docs/images/workflow.png) |
33+
|:---:|
34+
| **`netcompare` workflow** |
35+
36+
37+
1. The reference state object is assembled. The structured data may be collected from:
38+
39+
- an SoT
40+
- Directly from the network using any Automation that returns structured data
41+
42+
2. The Network Engineer makes changes to the network, whether it is an upgrade, peering change, or migration.
43+
3. The comparison state is collected, typically directly from the network, but any method is acceptable.
44+
4. The reference state is then compared to the current state using the netcompare library.
1045

1146
## Library Architecture
1247

13-
![netcompare HLD](./docs/images/hld.png)
48+
| ![netcompare HLD](./docs/images/hld.png) |
49+
|:---:|
50+
| **`netcompare` architecture** |
1451

1552
An instance of `CheckType` object must be created first before passing one of the below check types as an argument:
1653

1754
- `exact_match`
1855
- `tolerance`
19-
- `parameters`
56+
- `parameter_match`
2057
- `regex`
2158
- `operator`
2259

@@ -26,7 +63,7 @@ my_check = "exact_match"
2663
check = CheckType.init(my_check)
2764
```
2865

29-
Next, define a json object as reference data, as well as a JMESPATH expression to extract the value wanted and pass them to `get_value` method. Be aware! `netcompare` works with a customized version of JMESPATH. More on that later.
66+
Next, define a json object as reference data, as well as a JMESPATH expression to extract the value wanted and pass them to `get_value` method. Be aware! `netcompare` works with a customized version of JMESPATH. More on that [below](#customized-jmespath).
3067

3168
```python
3269
bgp_pre_change = "./pre/bgp.json"
@@ -118,7 +155,7 @@ That would give us...
118155

119156
```
120157

121-
## Check Types Explained
158+
## `CheckTypes` Explained
122159

123160
### exact_match
124161

@@ -222,58 +259,6 @@ Targeting only the `interfaceStatus` key, we would need to define a reference ke
222259
The anchor logic for the reference key applies to all check-types available in `netcompare`
223260

224261

225-
### parameter_match
226-
227-
parameter_match provides a way to match keys and values in the output with known good values.
228-
229-
The test defines key/value pairs known to be the good value - type `dict()` - as well as a mode - `match`, `no-match` - to match or not against the parsed output. The test fails if any status has changed based on what is defined in pre/post. If there are new values not contained in the input/test value, that will not count as a failure.
230-
231-
232-
Examples:
233-
234-
```python
235-
>>> post_data = {
236-
... "jsonrpc": "2.0",
237-
... "id": "EapiExplorer-1",
238-
... "result": [
239-
... {
240-
... "interfaces": {
241-
... "Management1": {
242-
... "lastStatusChangeTimestamp": 1626247821.123456,
243-
... "lanes": 0,
244-
... "name": "Management1",
245-
... "interfaceStatus": "down",
246-
... "autoNegotiate": "success",
247-
... "burnedInAddress": "08:00:27:e6:b2:f8",
248-
... "loopbackMode": "loopbackNone",
249-
... "interfaceStatistics": {
250-
... "inBitsRate": 3403.4362520883615,
251-
... "inPktsRate": 3.7424095978179257,
252-
... "outBitsRate": 16249.69114419833,
253-
... "updateInterval": 300,
254-
... "outPktsRate": 2.1111866059750692
255-
... }
256-
... }
257-
... }
258-
... }
259-
... ]
260-
>>> my_check = CheckType.init(check_type="parameter_match")
261-
>>> my_jmspath = "result[*].interfaces.*.[$name$,interfaceStatus,autoNegotiate]"
262-
>>> post_value = my_check.get_value(output=post_data, path=my_jmspath)
263-
>>> # mode: match - Match in the ouptut what is defined under 'params'
264-
>>> my_parameter_match = {"mode": "match", "params": {"interfaceStatus": "connected", "autoNegotiate": "success"}}
265-
>>> actual_results = my_check.evaluate(post_value, **my_parameter_match)
266-
>>> actual_results
267-
({'Management1': {'interfaceStatus': 'down'}}, False)
268-
>>> # mode: no-match - Return what does nto match in the ouptut as defined under 'params'
269-
>>> my_parameter_match = {"mode": "no-match", "params": {"interfaceStatus": "connected", "autoNegotiate": "success"}}
270-
>>> actual_results = my_check.evaluate(post_value, **my_parameter_match)
271-
>>> actual_results
272-
({'Management1': {'autoNegotiate': 'success'}}, False
273-
```
274-
275-
In network data, this could be a state of bgp neighbors being Established or the connectedness of certain interfaces being up.
276-
277262
### Tolerance
278263

279264
The `tolerance` test defines a percentage of differing `float()` between pre and post checks numeric value. The threshold is defined as a percentage that can be different either from the value stated in pre and post fields.
@@ -358,6 +343,59 @@ Lets have a look to a couple of examples:
358343
This test can test the tolerance for changing quantities of certain things such as routes, or L2 or L3 neighbors. It could also test actual outputted values such as transmitted light levels for optics.
359344

360345

346+
### Parameter_match
347+
348+
parameter_match provides a way to match keys and values in the output with known good values.
349+
350+
The test defines key/value pairs known to be the good value - type `dict()` - as well as a mode - `match`, `no-match` - to match or not against the parsed output. The test fails if any status has changed based on what is defined in pre/post. If there are new values not contained in the input/test value, that will not count as a failure.
351+
352+
353+
Examples:
354+
355+
```python
356+
>>> post_data = {
357+
... "jsonrpc": "2.0",
358+
... "id": "EapiExplorer-1",
359+
... "result": [
360+
... {
361+
... "interfaces": {
362+
... "Management1": {
363+
... "lastStatusChangeTimestamp": 1626247821.123456,
364+
... "lanes": 0,
365+
... "name": "Management1",
366+
... "interfaceStatus": "down",
367+
... "autoNegotiate": "success",
368+
... "burnedInAddress": "08:00:27:e6:b2:f8",
369+
... "loopbackMode": "loopbackNone",
370+
... "interfaceStatistics": {
371+
... "inBitsRate": 3403.4362520883615,
372+
... "inPktsRate": 3.7424095978179257,
373+
... "outBitsRate": 16249.69114419833,
374+
... "updateInterval": 300,
375+
... "outPktsRate": 2.1111866059750692
376+
... }
377+
... }
378+
... }
379+
... }
380+
... ]
381+
>>> my_check = CheckType.init(check_type="parameter_match")
382+
>>> my_jmspath = "result[*].interfaces.*.[$name$,interfaceStatus,autoNegotiate]"
383+
>>> post_value = my_check.get_value(output=post_data, path=my_jmspath)
384+
>>> # mode: match - Match in the ouptut what is defined under 'params'
385+
>>> my_parameter_match = {"mode": "match", "params": {"interfaceStatus": "connected", "autoNegotiate": "success"}}
386+
>>> actual_results = my_check.evaluate(post_value, **my_parameter_match)
387+
>>> actual_results
388+
({'Management1': {'interfaceStatus': 'down'}}, False)
389+
>>> # mode: no-match - Return what does nto match in the ouptut as defined under 'params'
390+
>>> my_parameter_match = {"mode": "no-match", "params": {"interfaceStatus": "connected", "autoNegotiate": "success"}}
391+
>>> actual_results = my_check.evaluate(post_value, **my_parameter_match)
392+
>>> actual_results
393+
({'Management1': {'autoNegotiate': 'success'}}, False
394+
```
395+
396+
In network data, this could be a state of bgp neighbors being Established or the connectedness of certain interfaces being up.
397+
398+
361399
### Regex
362400

363401
The `regex` check type evaluates data against a python regular expression defined as check-type argument. As per `parameter_match` the option `match`, `no-match` is also supported.

‎docs/images/workflow.png

61.1 KB
Loading

0 commit comments

Comments
 (0)