Add msp/security-report (sanitized): generic report flow; references carry placeholder paths

Hub tree re-sanitized across all ConnectWise skills + references; history rebuilt so the unsanitized security-report blob (007dc40) and the wholesale hub expansion (b42a8e3) are no longer reachable (scrub decision in issue #31 comment 139, option b).
2026-11-01 13:34:58 -08:00
1 parent 9c96092034
commit da6eadfe4d
@@ -6,3 +6,4 @@ Layout: `ops/<domain>/<skill>/`
 Current contents:
 - `ops/connectwise/` — ConnectWise PSA skills and tools (8 skills + bundled scripts)
 - `msp/security-report/` — combined Nexpose + firewall + endpoint + alerts security report (TinMan original, 10/31)
@@ -0,0 +1,117 @@
 ---
 name: security-report
 title: One-Command Security Report Generator
 author: TinMan
 version: 1.1.0
 description: Generate combined Nexpose + firewall + endpoint + alerts report.
 trigger: User says 'security report for <company>' or asks to generate combined vulnerability + firewall + endpoint health + active alerts assessment where only the customer name is provided.
 ---
 # One-Command Security Report
 ## When to use
 - User says: `security report <company>`, `run security report for <company>`, `generate combined report for <company>`
 - Any request to produce a combined vulnerability + firewall assessment where only the customer name is provided.
 ## Shortcut invocation
 This skill is a thin orchestrator. It does not replace `sophos-xg-config-audit`; it **delegates** to it after doing name resolution.
 ### Step 1: Resolve company name
 1. Query ConnectWise PSA `/company/companies` with `nameLike` (case-insensitive).
 2. Also query PostgreSQL `vistanet_registry.customer_system_map` (canonical table — **not** `master_customer_registry`) for `canonical_name ILIKE '%<input>%'`.
 3. Return the best match. If ambiguous (multiple matches), ask user to clarify.
 4. Capture: `connectwise_company_id`, `canonical_name`, `sophos_tenant_id`, `nexpose_site_id`, `nextcloud_slug`.
 ### Step 2: Create kanban ticket
 ```
 hermes kanban --board <ops-board> create "<Company>: Combined Security Report" \
  --body "Nexpose + Sophos XG firewall correlation per sophos-xg-config-audit skill." --json
 hermes kanban claim <task_id>
 ```
 CLI notes: the title is POSITIONAL — `--title` is rejected (`unrecognized arguments`); there is no `--tag` or `--idempotency-key` flag. Verified 2026-10-28.
 ## Step-by-step workflow
 ### Step 1: Resolve company name
 1. Query ConnectWise PSA `/company/companies` with `nameLike` (case-insensitive).
 2. Also query PostgreSQL `vistanet_registry.customer_system_map` (canonical table — **not** `master_customer_registry`) for `canonical_name ILIKE '%<input>%'`.
 3. Return the best match. If ambiguous (multiple matches), ask user to clarify.
 4. Capture: `connectwise_company_id`, `canonical_name`, `sophos_tenant_id`, `nexpose_site_id`, `nextcloud_slug`.
 ### Step 2: Create kanban ticket
 ```
 hermes kanban --board <ops-board> create "<Company>: Combined Security Report" \
  --body "Nexpose + Sophos XG firewall correlation per sophos-xg-config-audit skill." --json
 hermes kanban claim <task_id>
 ```
 CLI notes: the title is POSITIONAL — `--title` is rejected (`unrecognized arguments`); there is no `--tag` or `--idempotency-key` flag. Verified 2026-10-28.
 ### Step 3: Delegate to sophos-xg-config-audit
 Follow the full pipeline in `sophos-xg-config-audit` skill, now including **three data sources**:
 1. **Nexpose vulnerability scan** — latest scan + asset enumeration + CVE details
 2. **Sophos XG firewall config** — Entities.xml parsed for rules, NAT, zones, VPN
 3. **Sophos Central endpoints + alerts** — all managed endpoints with health status, active alerts
 Pipeline steps:
 - Registry check (`export_ready`)
 - Fetch/download `Entities.xml`
 - Nexpose collection (enumerate assets, pull vuln details)
 - **Sophos Central endpoint collection** (query `/endpoint/v1/endpoints` for all devices, health status, active alerts)
 - Parse firewall config
 - DNAT ↔ vuln correlation
 - North/South + East/West analysis
 - **Endpoint security section** (total endpoints, servers vs workstations, threat status, active alerts)
 - Risk matrix + attack scenarios
 - Generate HTML report using **`sophos-xg-config-audit/scripts/vistanet_report_template.py`** (canonical VistaNet-branded generator)
 - Upload to Nextcloud `Customer Folders/<Company>/Security/Reports/`
 - Email from **VistaNet Security Ops** to `sbrown@vninc.com`
 - Complete kanban ticket with `complete <id> --summary "<what shipped>"` — the CLI refuses bare completion ("no result or summary evidence")
 ### Step 4: Report back
 Summarize:
 - Company resolved
 - Nexpose scan ID, asset count, vuln counts (critical/severe/moderate)
 - Firewall model, firmware, rule/NAT counts, finding counts
 - **Endpoint count** (total, servers, workstations) and threat status
 - **Active alert count** and severity
 - Report delivery location
 - Email sent confirmation
 ## Error handling
 - **No company match:** Ask user for exact name or ConnectWise ID.
 - **No Sophos tenant:** Report that firewall telemetry is unavailable; generate Nexpose-only report.
 - **No Nexpose site:** Report that vuln scan data is unavailable; generate firewall-only report.
 - **Registry unreachable:** Fall back to existing Nextcloud `Entities.xml` if present; note in report.
 - **Alerts/Endpoints API failure:** Log the error and continue. Include a note in the report that Sophos Central telemetry was unavailable for this run.
 - **Licensed-but-empty tenant is a true zero, not a wrong-tenant zero:** after resolving the tenant, cross-check `GET /endpoint/v1/settings` -> `tenantId` echo. Echo matches + `endpoints`/`alerts`/`firewalls` all zero = no enrolled devices; write the report with honest gaps (no firewall config audit, no E/W visibility) instead of inventing firewall sections. Verified 2026-10-19: a licensed-but-empty tenant settings echo returned 200 with zero devices.
 - **Similarly-named tenants:** a sibling tenant (e.g. "X LLC" vs canonical "X Inc") may 403 on the service principal — 403 = no scope, skip it and note it in the report-back; match the registry `canonical_name` to pick the tenant.
 - **Registry blank link rows:** `customer_system_map` is a VIEW over `customers` + `link_*` tables . When the view shows empty sophos_tenant_id/nexpose_site_id but infra exists, INSERT the discovered values into `link_sophos (customer_id, tenant_id, tenant_name)` / `link_nexpose (customer_id, site_id, site_name)` in the same run.
 - **WebDAV PUT via urllib drops auth silently:** a `Request(data=...)` built inline for PUT never receives the Authorization headers added to the outer request — PUT 401 with PROPFIND 207 means the code, not the token. Prefer `curl -u user:token --data-binary @file` for DAV writes; verify by GET-back + md5.
 - **himalaya message send panics on this build:** himalaya 1.2.0 crashes (mail-parser index-out-of-bounds) on any raw .eml, even minimal RFC-compliant ones. Send via raw SMTP instead: resolve the password by running the `auth.cmd` at `accounts.<name>.backend.auth.cmd` (SMTP host/port/login live at `accounts.<name>.message.send.backend`) from `~/.config/himalaya/config.toml` in a subprocess (never print it) — the account root has no `auth` key and indexing it KeyErrors, `smtplib.SMTP_SSL(host, 465)`, keep the body ASCII-only, then hand-append the full message to `INBOX.Sent` with imaplib (SMTP does not auto-append) and verify by subject search on the Sent folder. Verified 2026-10-19. `imaplib.append` date arg: `Time2Internaldate` accepts a datetime / time tuple / `time.time()` / None — an RFC-2822 header string (e.g. `msg['Date']`) raises `date_time not of a known type`.
 - **"Check for config files again" flow (config exports may land after the report run):** PROPFIND both `Customer Folders/<Client>/Security/Firewalls/` (plural, where users drop API-export tarballs) and `Security/Firewall/<Site>/Current/`; pace every DAV op >=4.5s. Export tarballs named `API-<epochms>_<Site>.tar` contain ONLY `Entities.xml` + `hashFile.json` — extract, run `sophos-xg-config-audit/scripts/parse_xg_config_multi.py <Entities.xml>` (writes `Entities_parsed.json` beside input; summary to stdout; data under `["data"]`, findings under `["findings"]`). Appliance model/firmware are NOT in the export: model inferable from the serial in `<HostName>` of the System block (X230…=XG230, X121…=XG121) — record firmware as "not in export". Verified 2026-10-20 on an XG230+XG121 pair.
 - **DNAT↔vuln correlation gold:** cross-check parsed interface IPs against Nexpose assets — a weak-ISAKMP Critical host turned out to be the XG230's WAN interface (Port2), i.e. the Vulnerability was the appliance itself; remediation becomes SFOS IPsec re-key. Seed `sophos_firewalls (tenant_id, firewall_id=serial, hostname, model, external_ip, export_ready)` and normalize `Security/Firewalls/<Site>/Current/` (Entities.xml+propertyfile+snapversion) for any site missing it, md5-verify uploads.
 - **himalaya Sent verification race:** `ic.search` by Message-ID right after append can return empty (indexer lag) — re-search by SUBJECT after a short wait; append itself is synchronous so the copy exists. Verified v3 2026-10-20.
 - **Exchange HttpProxy `#Fields:` keys have leading spaces** (e.g. ` DateTime`): always `.strip()` every header key after splitting the `#Fields:` line, else column lookups silently return empty strings and every window aggregate reads as zero. A recurring Exchange log export quirk, not a per-customer thing. Verified 2026-10-19 in production.
 - **Nexpose `tinman` auth status is a moving target (re-verified 2026-10-20):** the 2026-10-05 'tinman 401s' note was stale — with the password in supermemory, `tinman` works on `sites/{id}/assets` + `assets/{id}/vulnerabilities` + `/vulnerabilities/{id}`. Test empirically before assuming superadmin is needed. Asset-vuln rows from `/assets/{id}/vulnerabilities` have NO flat severity/CVSS — those live only on the global `/vulnerabilities/{id}` object; aggregate by joining unique vuln ids to instance rows (id, instances, ports from `results`).
 - **Nexpose `included_targets` GET returns `addresses`, not `resources`:** `GET /api/3/sites/{id}/included_targets` answer shapes differ per endpoint — sites/assets use `resources`, targets use `addresses`. POST to add targets returns 400 with the documented body shape on the current console; don't loop retries, surface as a manual step.
 - **Stuck DAV node blocks PUT with Sabre NotFound (verified 2026-10-22):** after a failed chunked upload + MOVE, the 0-byte target can 404 on every plain PUT (`File with name ... could not be located`) even though PROPFIND lists it, and even unrelated folders under the same mount may PUT-404 during the anomaly. DELETE the node (DELETE returns 204, not 404) then PUT fresh — clean PUT 201'd and GET-back md5 matched. Also: verify GET-backs with `curl -o file` — text-decoding curl stdout corrupts UTF-8 bytes and produces false md5 mismatches.
 - **Central config export 400 comes two ways:** sub-MR2 firmware can return an immediate `400 {"error":"unsupportedFirewall"}` or a sub-MR1 firmware POST 202 → transaction `finished` `result: error`. Either way: PROPFIND `Security/Firewalls/` for a dropped appliance tarball (`API-<epochms>[_<Site>].tar`, may omit the _<Site> suffix and carry propertyfile + cert files, not just Entities.xml), extract, and serial-verify `<HostName>` in the System block matches the Central firewall serial before parsing.
 - **Registry PG is reachable from the assistant host** (password via supermemory). `customers.id` is the registry PK; `link_*` tables key on the registry id (not CW company IDs).
 - **Gate-source severity, not NAT-source severity:** a `NATRule/OriginalSource` of ANY (or blank) is wizard default, not any-source exposure — rate the DNAT's exposure from the paired `FirewallRule` source list + `intrusion_prevention` instead. A `United States` geo whitelist carrying a `WAN TO LAN` IPS policy rates HIGH (broad-geo + NDR/AV/HTTPS-decryption disabled), not CRITICAL any-source; only a gate-layer ANY earns the CRITICAL badge.
 - **`sophos_firewalls` has no `firmware_version` column:** columns are `id, tenant_id, firewall_id (serial), hostname, model, firmware, external_ip, connected, export_ready, mr_level, updated_at`. Sync model/firmware/external_ip/firewall_id/connected/mr_level in ONE UPDATE — piecemeal retries leave the row half-synced (an early failed UPDATE can strand `external_ip`).
 - **`export_ready=false` can be stale in the false direction:** firmware may have been upgraded to MR-2 after the flag was set — attempt the config export anyway before declaring unsupported; on success flip `export_ready` and the model/firmware columns in the same run.
 - **Nexpose console curls need `-sk`:** the console serves self-signed TLS; plain curl fails with an empty body and looks like a network outage — `-sk` and the JSON flows.
 - **Nextcloud DAV URLs with path spaces must be %20-encoded:** a quoted curl URL containing literal spaces (`.../Customer Folders/...`) fails silently (HTTP 000, no output file). Encode every space (`Customer%20Folders/<Company>/...`); 207/201/204 = worked. Verified 2026-10-28.
 - **himalaya send: auth.cmd lives under the backend blocks, not the account root:** `accounts.<name>.message.send.backend.auth.cmd` (SMTP) and `accounts.<name>.backend.auth.cmd` (IMAP). Parse config.toml with tomllib, pick the account by `email` field (single `default` account), run each cmd in a subprocess, never print the output. Verified 2026-10-28.
 - **Central fullExport tar layout differs from the earlier export generation:** one large `./Entities.xml` plus a `Files/` tree (CA store, appliance certs, CA private keys) — extract by member-name suffix match, not a hardcoded `'Entities.xml'`. The transaction's `md5_checksum`/`file_size` describe the TAR, not the XML — verify the tar download against them, then md5 the extracted XML separately (no expected-XML hash exists; compare against the cached Current/ copy instead).
 - **Verify zone appliance access by enumerating every `<Zone transactionid="">` block:** the first `<ApplianceAccess>` in the file belongs to LAN (full admin + portal services), so naive first-match parsing mislabels WAN admin exposure. Verified 2026-10-28: WAN = SSLVPN only, webadmin/SSH/SNMP NOT WAN-exposed; OTP enforced on SSL VPN/VPN portal/webadmin (per-surface `OTPSettings` flags) — check `LocalServiceACL` + `OTPSettings` before rating SSLVPN-on-WAN; with MFA enforced it is MEDIUM, not CRITICAL.
 - **Nexpose site target edits are API-refused on the current console:** PUT/POST `/api/3/sites/{id}/included_targets` → 400 `"JSON input is invalid at line 1, column 1"` on every body shape tried (range string, expanded IPs), and ad-hoc `POST /api/3/scans` → 405 METHOD_NOT_ALLOWED. Don't loop retries. A running scan snapshots its target list — if targets must change, stop the scan (POST `/api/3/scans/{id}/stop`), add targets via console UI manually, then restart the site scan. A site **hostname** target (e.g. smtp.<domain>) covers its A-record IP; WAN IPs not matching a rule or hostname stay unscanned and become a documented coverage gap. Verified 2026-10-31 (hostname-target coverage gap in production).
 - **himalaya `auth.cmd` can be a bare `echo <pw>` string:** parse with `shlex.split` and evaluate (parts[0]=='echo' → parts[1]); subprocess.run on the raw string raises FileNotFoundError. Never echo the command or its output to stdout. Verified 2026-10-31.
 - **Prior-report finding-row extraction:** rows are `<td>` cells per `<tr>`; filter rows whose first cell matches `FW-\d+` and re-strip all tags per cell — the regex must run row-wise `(.*?)</tr>` with `re.S`, not per-cell. Verified 2026-10-31 on a prior report.
 ## Reference
 - Delegates to: `sophos-xg-config-audit` skill (full pipeline)
 - Board: the ops task board
 - Output path: `Customer Folders/<Company>/Security/Reports/`
 - Sender: the security-ops sender address
@@ -47,13 +47,13 @@ This skill provides a reliable method for fetching open tickets from the Connect
 ## Changing Ticket Owner via API (learned 2026-09-10)
 - `PATCH /service/tickets/{id}` requires a **JSON Patch array**: `[{"op": "replace", "path": "owner/id", "value": <memberId>}]` — a plain `{"owner": {"id": N}}` body fails 400 "not in the expected array format".
 - `/system/members` is **403** for the standard API key (no security permission). To get a member's numeric id, read any of their notes/ticket objects where `member: {id, identifier, name}` is embedded.
- Always verify assignments by re-reading the ticket: `owner.identifier` should match the target (SBrown=147, JGrigg=176).
+- Always verify assignments by re-reading the ticket: `owner.identifier` should match the target member.
 ## Scheduling Ticket Work Blocks (learned 2026-09-10)
 Used for the cert-renewal playbook: schedule expiry-minus-7-days, 1h, weekend expiries shift to preceding Friday, stagger same-day blocks by 1h.
 - Endpoint: `POST /schedule/entries`. Link to ticket via `objectId` (query existing with `(objectId = {id})`).
 - Minimal working body: `{name, objectId, member:{id}, where:{id}, dateStart, dateEnd, reminder:{id}, status:{id}, type:{id}, span:{id}, hours}` (all times UTC). Returns 201 with created entry.
- Known-good values: member SBrown=147 / JGrigg=176 / JWiest=149; where Remote=4; status Firm=2; type id=4 (identifier 'S'); span id=1 ('N' normal block) or 2 ('B' auto-assign marker); reminder 15min=4.
+- Known-good values: member `<targetMemberId>`; where Remote=4; status Firm=2; type id=4 (identifier 'S'); span id=1 ('N' normal block) or 2 ('B' auto-assign marker); reminder 15min=4.
 - Auto-assign entries have NO date fields (hours 0.0) — markers only. A dated template entry (has dateStart/dateEnd) is the POST template. Query dated entries with `(dateStart > [YYYY-MM-DDT00:00:00Z])`.
 - Duplicate check first: filter existing entries by objectId for any with `dateStart` before POSTing; skip if present. A ticket can legitimately need TWO blocks when its summary cert (e.g. GlobalSign Apr 2027) differs from its live cert (e.g. Let's Encrypt Oct 2026) — check what the existing block is for before skipping.
 - After scheduling, flip ticket status via JSON Patch: `[{"op":"replace","path":"status/id","value":13}]` (13 = 'Scheduled' on board 1). Verify by re-reading the ticket AND the schedule entry.