Design a security findings collector that does not hide failure

Security automation often reads findings from several services and produces a report. Its correctness is part of security. If a collector stops after the first page or treats an API error as an empty list, it can tell the team that no findings exist while important records were never read.

Define the output's completeness contract. A report should identify source, organization, collection time, pagination completion and any failed scope. Distinguish zero findings from unknown findings. If one source fails, the tool can return partial results with explicit status or fail the whole report under policy. It should not silently publish a green total.

Use the least authority needed. A read-only findings collector should not receive administrative write credentials merely because they are easier to obtain. Bind organization scope to trusted configuration and enforce it in requests. Avoid printing tokens in command lines, logs or error dumps.

Retries need bounded behavior. Respect source rate limits and use backoff under the documented API contract. A retry after a page request should not duplicate results or skip a cursor. Store stable finding IDs and retain source identity when combining systems so unrelated IDs do not collide.

In an interview, show a page trace and a report result for partial failure. This is more convincing than saying the tool is robust. The learner should be able to explain what the tool knows, what it missed and what the operator can do next.

Teaching tool collects vulnerability findings for one configured organization from a paginated API. Each page contains findings, a next_cursor and a source_snapshot identifier. A finding has source ID, organization ID, status and severity. The API token is scoped to read findings for that organization. These fields define an invented API for the exercise; they are not a claim about any vendor's exact schema.

The snapshot field matters if findings change during collection. Without a consistent snapshot, pages can overlap or shift. The tool may still provide a useful best-effort report, but it should state that limit and deduplicate by stable source, organization and finding identity. If the source supplies no snapshot mechanism, do not invent one in the client or claim a perfectly atomic inventory.

Page one returns finding F1 and cursor C2. Page two returns F2 and F3 with cursor C3. Page three returns a rate-limit response. The naive implementation catches the error and returns the three collected findings as a complete report. The correct implementation keeps collection status incomplete and follows the bounded retry policy. If retry succeeds with F4 and no next cursor, the final complete set is F1–F4.

If the retry repeats page two by mistake, stable composite IDs can prevent duplicate F2 and F3 rows, but the tool may still fail to reach page three. Deduplication does not prove pagination completeness. Record the last successful cursor and the terminal condition separately.

A partial report can say 'three findings collected; source pagination incomplete at cursor C3; last successful collection at 14:02.' Do not expose the raw cursor if it contains sensitive data; a safe operation reference may be enough. The operator needs a retry path and a clear warning that totals are not complete. The user interface should not color the organization green because the partial count happens to be zero.

Cache behavior needs the same honesty. If a source is unavailable, the tool may display the last complete report with its age and an explicit refresh failure. It should not relabel old findings with the current time. A stale report can still help an operator, provided its status is clear.

A test suite can use a local fake API that returns several pages, repeats an item, emits a temporary error and finally completes. Assert unique source/organization/finding IDs and the completeness flag. Add a permanent failure case and verify the report remains partial or fails under policy. Add a wrong-organization response and reject it rather than mix data into the configured scope.

Credential handling should use the environment's approved secret mechanism. Avoid passing the token as a visible command argument in environments where process listings expose it. Redact authorization headers in errors and tests. The tool's logs should help correlate a source request without creating another credential leak.

Contrast Security publishes an external public challenge asking an Integrations Developer to build a CLI around its API. That supports the practical relevance of API-client work, but this security findings collector, its pagination schema and failure cases are original practice. Do not tag this exact prompt as a verified Security Engineer question at Contrast. The original role and unknown current use remain in the evidence record.

The interview answer ends with a sample partial output and a complete output. A security collector's most dangerous failure can be false reassurance. Explicit uncertainty is a functional requirement, not a cosmetic warning added after the report is built.

Teaching report artifact:

1organization: red2source: scanner-a3collection_status: partial4findings_collected: 35last_complete_report: 2026-09-20T14:00Z6current_error: pagination_retry_exhausted7security_state: unknown_for_uncollected_pages

The report does not replace missing data with zero. A later successful run records four unique findings and complete status. These invented outputs can be checked locally without credentials or an external account.

Two sources each return finding ID 17. Source A completes with no further pages. Source B fails after page one. How should identity and report status work? Give one test that catches false reassurance.

Use source, organization and finding ID as the durable identity, such as A:red:17 and B:red:17. This preserves the configured organization when outputs are later combined. Mark B incomplete and the aggregate report partial if both sources are required. Keep A's completed results with their source status. Test a failed B page when its first page is empty and assert that the report says unknown/incomplete, not zero findings and healthy. Verify retry exhaustion produces a non-success or explicit partial outcome under the contract.

Score out of four: one point for the correct result, one for showing the intermediate reasoning, one for identifying the stated failure case, and one for a verification that could disprove the answer. Do not award the reasoning point for a tool name alone.

Misconception 1: deduplicated rows imply a complete collection. Pagination can stop early even with unique output. Misconception 2: an API error is equivalent to an empty result. Unknown scope must remain visible or the tool can create false security assurance.

Evidence class: recommended. Original practice.

How would you make a vulnerability-reporting CLI trustworthy?

Strong answer: I would define source scope, least-privileged credentials, pagination completion, stable IDs, bounded retries and explicit partial results. I would test empty-first-page failures and avoid treating missing data as no findings.

Follow-up: What can you promise if the source API offers no consistent snapshot across pages?

Weak answer indicators: Catching all errors as empty arrays; token values in logs; source-ID collisions; stale reports shown as current.

Technical references: Contrast Security public challenges; OWASP secrets management; OWASP logging. Sources support the documented mechanisms. The numbers, decisions, rubrics and interview prompts in this lesson are original teaching examples, not measurements or employer question claims.