Skip to content

Improve scanner features and introduce web dashboard with API#32

Merged
kbtale merged 7 commits into
mainfrom
dev/fixes
May 10, 2026
Merged

Improve scanner features and introduce web dashboard with API#32
kbtale merged 7 commits into
mainfrom
dev/fixes

Conversation

@kbtale

@kbtale kbtale commented May 10, 2026

Copy link
Copy Markdown
Member

Python Version Compatibility and CI Updates

  • Lowered the minimum required Python version from 3.14 to 3.10 in pyproject.toml, .python-version, and README.md, and updated dependencies to fix compatibility with Python 3.10+. The CI workflow (.github/workflows/python-ci.yml) now tests across Python 3.10–3.14 [1] [2] [3] [4] [5] [6]

Port Scanning and Service Detection

  • Integrated a new TCP port scanning phase in the discovery pipeline by importing and utilizing TcpPortScanner and classify_host_services. The scan results (open ports and identified services) are now collected for each discovered host. [1] [2]
  • Extended the discovery summary to include detailed TCP port scan data, target/result counts, and a list of detected services, ensuring these results are available for reporting and downstream processing. [1] [2] [3]

Database and Reporting

  • Modified the database observation logic to persist service detection results (service_data) as part of the scan summary.

Documentation and Roadmap

  • Added a comprehensive critical fixes and implementation roadmap (Fixes-roadmap.md), detailing plans for raw socket ICMP scanning, web dashboard, port scanning, Python version support, and MAC address resolution, along with execution order and success criteria.

kbtale added 7 commits May 9, 2026 13:46
- Lower requires-python to >=3.10 for real-world adoption
- Add multi-version CI matrix (3.10 through 3.14)
- Fix .gitignore to allow Fixes-roadmap.md
- Add comprehensive audit and implementation roadmap
… sweep

- _parse_arp_table now returns {ip: mac} dict instead of bare IP list
- Added _normalize_mac helper for canonical colon-separated MAC format
- Refreshes ARP table after ICMP sweep to capture newly-learned MACs
- Added test_arp_parser.py with 14 unit tests for parsing and normalization
- Updated existing test mocks for new return type
… ping)

- New src/backend/transports/icmp.py: IcmpScanner using SOCK_RAW/IPPROTO_ICMP
- Batch send/receive with select() loop; ~50-100x faster than fork-per-IP
- Automatic fallback to subprocess ping when raw sockets unavailable (non-admin)
- Correct RFC 792 checksum computation in network byte order
- Benchmark tool at tools/benchmark_scan.py compares old vs new
- 18 unit tests for checksum, packet construction, raw/subprocess paths
- Integrated into _python_ping_sweep replacing ThreadPoolExecutor ping loop
- New src/backend/transports/tcp_scan.py: TcpPortScanner with configurable
  port list (13 default ports), connect() per port, banner grab
- New src/backend/transports/fingerprint.py: service identification from
  port numbers and banner heuristics (SSH, HTTP, FTP, SMTP, RDP, SMB, etc.)
- Integrated into discovery pipeline as a new phase after ICMP sweep
- TCP results fed into service_identity pipeline via service_data
- Open ports displayed in TUI host details panel
- Database persistence for TCP service evidence rows
- 23 unit tests covering scanner, fingerprint, banner matching
- New src/presentation/web/server.py: stdlib HTTP server with endpoints
  GET  /api/devices, /api/summary, /api/scan/status, /api/export/*
  POST /api/scan/start (triggers background discovery with progress polling)
- New src/presentation/web/dashboard.html: zero-dependency SPA
  Dark theme, sortable device table, search/filter, copy IP, RDP/HTTP actions
  Scan control with live progress bar, CSV/JSON export buttons
- New 'web'/'serve' CLI subcommand (netdocit web) opens browser automatically
- 11 unit tests covering all API endpoints and handler routing
networkx 3.5+ requires Python 3.11, breaking CI on the 3.10 runner.
No API surface loss: all used APIs (Graph, add_node, add_edge, nodes,
neighbors, get_edge_data) exist since networkx 2.x.
pyright reports dict.get(None, ...) where dict is typed dict[int, str].
@kbtale kbtale merged commit 368968c into main May 10, 2026
2 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant