Differences

This shows you the differences between two versions of the page.

--- syntheses:customer-side-bug-finding-methodology [16/07/2026 03:54] – created - external edit 127.0.0.1
+++ syntheses:customer-side-bug-finding-methodology [15/10/2026 05:24] (current) – removed - external edit (Unknown date) 127.0.0.1
@@ Line 1: / Line 1: @@
-====== Customer-Side Bug Finding Methodology ======
-===== Notes =====
-===== Summary =====
-====== Customer-Side Bug Finding Methodology ======
-===== Purpose =====
-Identify bugs that a **customer** (authenticated user of the panel, API, or CLI) can trigger, from a customer's perspective. NOT internal admin bugs or ops issues.
-===== Rule: External vs Internal =====
-**EXTERNAL (customer-triggerable):**
-  * Any endpoint in WebApi, PanelSite, or ClassicPanel that accepts customer input
-  * Any validation layer that processes customer-provided data
-  * Any domain entity that customers can modify (Guest, FirewallRule, DomainName, etc.)
-  * The chain: Customer → WebApi/Panel/CLI → Domain validation → NHibernate → HostCommand → HostDaemon → iptables
-**INTERNAL (not customer-triggerable):**
-  * AdminSite (staff-only)
-  * HostDaemon running on compute hosts (customers can't SSH to these)
-  * StatsDaemon, Icinga, Tron, Ansible
-  * Internal ops scripts and provisioning tooling
-  * Third-party forks
-===== How to Find Customer-Triggerable Bugs =====
-==== Pattern 1: Unbounded Input Collection ====
-Search for ''foreach'' loops over customer-controlled collections without ''.Take()'' or count caps.
-  * **Example found:** GuestFirewallRule.GetIptablesEntries() — 4-level nested loop over source/dest/protocol/port with no bounds
-  * **Where to look:** Every POST/PUT endpoint that accepts a ''List<T>'' from customer input
-==== Pattern 2: Per-Item Validation Without Per-Collection Validation ====
-Search for validators that check individual items but not the collection itself.
-  * **Example found:** ValidateAddressListAttribute validates each address but not how many addresses are in the list
-  * **Where to look:** Domain/Validators/ directory, especially attributes used on collection properties
-==== Pattern 3: Cartesian Product Expansion ====
-Search for nested ''foreach'' loops that multiply input sizes.
-  * **Formula:** If input A has N items and input B has M items, the loop runs N×M times
-  * **Where to look:** Any method that iterates over multiple customer-provided collections
-  * **Key file:** GuestFirewallRule.cs lines 78-86 (source × destination × protocol × port)
-==== Pattern 4: Lock Hold During Expensive Operations ====
-Search for ''lock'' statements or claim_lock patterns that hold during long-running operations.
-  * **Example found:** qemu-network/mammothvps.sh uses claim_lock "iptables" held during the entire apply phase
-  * **Impact:** One customer's slow operation blocks all other guests on the same host
-  * **Where to look:** HostDaemon, qemu-network scripts, any code with "lock" or "claim"
-==== Pattern 5: Missing Idempotency on Mutations ====
-Search for POST endpoints that don't check for duplicate execution.
-  * **Impact:** Double-clicking "reinstall" or "restart" could cause unexpected state
-  * **Where to look:** WebApi controllers, ClassicPanel POST endpoints
-==== Pattern 6: Race Conditions in Multi-Step Transactions ====
-Search for operations that span multiple steps without atomic guarantees.
-  * **Examples found:** ApproveAccount/ProcessDomainNameCommand race (#1635), PayPal register-then-capture (#1863)
-  * **Where to look:** Domain/Commands/ files, especially those handling money or domains
-==== Pattern 7: String Parsing Without Size Limits ====
-Search for Split(), Parse(), Deserialize() on customer input without size checks.
-  * **Impact:** Memory exhaustion from oversized input
-  * **Where to look:** GuestFirewallRule.FromJson(), FromDictionary(), any endpoint accepting raw JSON strings
-==== Pattern 8: Missing Null/Default Checks on Nullable Collections ====
-Search for ''.Count'' or ''.Any()'' on collections that could be null.
-  * **Where to look:** Entity classes, especially those with virtual collections
-===== Quick Search Commands for Bug Hunting =====
-From the vps/vps repo, search for these patterns:
-<code bash>
-# Unbounded foreach loops in customer-facing code
-grep -rn "foreach.*FirewallRule\|foreach.*guest\|foreach.*customer" product/website/Domain/ product/website/WebApi/ product/website/ClassicPanel/
-# Per-item validation without count limits
-grep -rn "ValidateAddressList\|ValidationResult" product/website/Domain/Validators/
-# Nested loops (potential Cartesian products)
-grep -rn "foreach.*foreach" product/website/Domain/Entities/
-# Lock/claim patterns
-grep -rn "claim_lock\|lock.*iptables\|lock.*guest" product/website/HostDaemon/ infrastructure/
-# Missing .Take() or count caps on collections
-grep -rn "\.Add(.*rule\|\.Add(.*customer" product/website/Domain/Services/
-# POST endpoints without idempotency
-grep -rn "Verb.Post\|HttpPost" product/website/ClassicPanel/Controllers/ product/website/WebApi/Controllers/ | grep -v "Get\|Idempotent\|Check"
-</code>
-===== High-Value Targets for Customer-Side Bug Hunting =====
-==== Firewall Rules (highest ROI) ====
-  * **Attack vector:** Submit massive firewall ruleset via API or ClassicPanel import
-  * **Impact:** Stalls all guests on the same compute host (known issue #1640)
-  * **What to test:** Create a guest, add 1000+ firewall rules with 100+ addresses each via API. Monitor host load and other guest network operations.
-==== Guest Creation/Provisioning ====
-  * **Attack vector:** Create many guests simultaneously, or create with unusual configurations
-  * **Impact:** Race conditions in provisioning (#1863), stale state (#1884)
-  * **What to test:** Concurrent guest creation with same payment method, unusual OS/plan combinations
-==== Domain Registration ====
-  * **Attack vector:** Register domain + create account simultaneously
-  * **Impact:** ApproveAccount/ProcessDomainNameCommand race (#1635, #1863)
-  * **What to test:** Simultaneous account creation with domain registration, rapid cancel/re-register
-==== Payment/Billing ====
-  * **Attack vector:** Modify payment method while Mpay is slow/unavailable
-  * **Impact:** Token-delete timeout (#1811), billing on wrong card (#1891)
-  * **What to test:** Kill Mpay service mid-transaction, rapid card add/Remove cycles
-==== Guest Power Operations ====
-  * **Attack vector:** Rapid start/stop/reinstall cycles
-  * **Impact:** State corruption during HostCommand processing
-  * **What to test:** Power cycle guest 10x rapidly while it's being migrated
-==== DNS/Forward DNS ====
-  * **Attack vector:** Add many DNS records, or malformed DNS entries
-  * **Impact:** DNS propagation failures, record loss
-  * **What to test:** 500+ DNS records per domain, unusual record types
-===== Testing Strategy =====
-  - **Isolated environment:** Use a single guest on a known host (check via API which host a guest is on)
-  - **Observe side effects:** Monitor OTHER guests on the same host during your test
-  - **Measure, don't just observe:** Note response times, error rates, CPU usage
-  - **Reproduce deterministically:** If it happens once, can you make it happen again?
-  - **Document:** Record exact steps, inputs, and observed behavior
-===== What NOT to Test =====
-  * Anything requiring admin panel access
-  * Anything requiring SSH to compute hosts
-  * Production customer data
-  * Anything that could cause data loss without approval
-  * Database schema changes
-===== Related =====
-==== Related Pages ====
-  * [[binarylane-architecture-panels-customer-entry-points|BinaryLane Architecture - Panels & Customer Entry Points]]