Differences

This shows you the differences between two versions of the page.

--- syntheses:vps-create-flow [17/07/2026 03:15] – external edit 127.0.0.1
+++ syntheses:vps-create-flow [03/08/2026 23:16] (current) – removed - external edit (Unknown date) 127.0.0.1
@@ Line 1: / Line 1: @@
-====== VPS Creation / Provisioning Flow ======
-This page traces the source-backed flow for creating a VPS from the customer/API request through to HostDaemon provisioning on the compute host.
-===== Scope =====
-  * Main repo: ''vps/vps'' project 1
-  * Main application area: ''product/website''
-  * Compute execution: HostDaemon and host command handlers
-  * Related repo: ''vps/compute'' project 22, for deployed compute-host programs and scripts
-===== Short Version =====
-  - The API accepts a create-server request and validates the customer, plan, OS, network, billing state, and requested options.
-  - Domain services create a ''Guest'' in the database with status ''Building''.
-  - Provisioning services assign the lease, host, network, primary interface/IPs, backup schedule, and default firewall state.
-  - ''GuestManagementService.CreateGuest()'' creates a ''CreateGuestCommand''.
-  - HostDaemon polls the ''HostCommand'' table, marks the command started, and runs ''CreateGuestCommandHandler''.
-  - On success, HostDaemon sets the guest status to ''Active'', commits the guest, records system actions/host logs, sends mail, updates network members, and marks the command completed.
-  - On failure, HostDaemon attempts cleanup/refund/cancellation and leaves the guest suspended or terminated depending on where the failure occurred.
-===== API and Domain Flow =====
-==== 1. API request ====
-''product/website/WebApi/Services/GuestPlanApiService.cs''
-''GuestPlanApiService.CreateServer()'':
-  * fetches the customer via ''CustomerResourceService.GetCustomer()'';
-  * checks ''customer.Subscribed'', except where signup flow allows otherwise;
-  * validates purchase eligibility, guest limit, DTO fields, and active/listed plan;
-  * resolves creation parameters such as facility, plan, operating system, network, and extras;
-  * creates the guest/domain object inside a transaction;
-  * flushes NHibernate so the API ID exists;
-  * attempts immediate invoice processing when the blocking invoice requires payment;
-  * returns an action DTO for the created host command.
-Relevant source notes:
-  * ''GuestPlanApiService.CreateServer()'' calls ''GuestCreationService.CreateNewGuestDomain()'' inside ''TransactionRunner.Transaction()''.
-  * If ''command.BlockingInvoice().RequiresPayment()'', it calls ''InvoiceService.TryProcessInvoice(blockingInvoice, null)''.
-==== 2. Guest/domain creation ====
-''product/website/Domain/Services/GuestCreationService.cs''
-''GuestCreationService.CreateNewGuestDomain()'':
-  * repeats purchase and guest-limit checks at domain level;
-  * validates private IP/VPC details;
-  * creates the guest entity through ''GuestFactory.InitializeNewGuest()'';
-  * validates the guest;
-  * applies options such as port blocks, IPv6, split NICs, SSH keys, script/user-data inputs, and duration;
-  * calls ''GuestProvisioningService.ProvisionNewGuest()'';
-  * commits the guest;
-  * handles referral/fraud-review side effects;
-  * queues the host command through ''GuestManagementService.CreateGuest()''.
-==== 3. Initial guest state ====
-''product/website/Domain/Services/GuestFactory.cs''
-''GuestFactory.InitializeNewGuest()'' sets up the in-memory/new database guest and calls:
-<code>
-guest.SetStatus(GuestStatus.Building);
-</code>
-It also sets initial host/storage/CPU placeholders and applies the selected operating system and plan.
-==== 4. Provisioning assignments ====
-''product/website/Domain/Services/GuestProvisioningService.cs''
-''ProvisionNewGuest()'' performs the pre-daemon allocations:
-  - ''GuestLeaseCreationService.CreateNewLeaseForGuest()'' creates the lease and lease term.
-  - ''HostService.AddGuestToAvailableHost()'' assigns a compute host.
-  - ''NetworkService.AddGuestToNetwork()'' assigns a network.
-  - ''GuestService.AssignPrimaryInterfaceToGuest()'' assigns the primary interface/IP and commits the guest.
-  - ''GuestService.AssignAdditionalInterfaces()'' handles extra interfaces/VPC NICs.
-  - ''BackupScheduleService.SetupGuestBackupSchedule()'' sets backup schedule state.
-  - ''SetupDefaultFirewall()'' clears/initialises firewall rules.
-===== HostCommand / HostDaemon Flow =====
-==== 5. CreateGuestCommand is queued ====
-''product/website/Domain/Services/GuestManagementService.cs''
-''CreateGuest()'' creates a ''CreateGuestCommand'', attaches password/SSH/script/user-data inputs, and calls ''HostCommandService.AddHostCommand()''.
-''product/website/Domain/Entities/CreateGuestCommand.cs''
-The command has six visible steps:
-  - Requesting server resources
-  - Allocating virtual disk
-  - Installing operating system
-  - Starting server
-  - Configuring operating system
-  - Processing invoice
-It implements ''IInvoiceBlockableCommand''. Its blocking invoice is:
-<code>
-Guest.LatestLeaseTerm.Invoice
-</code>
-==== 6. HostDaemon picks up the command ====
-''product/website/HostDaemon/Hosted/HostCommandThread.cs''
-HostDaemon repeatedly:
-  - asks ''HostCommandBroker.GetNextCommand()'' for the next not-started, not-completed, unblocked command;
-  - calls ''HostCommandService.TryStartCommand()'';
-  - starts ''ProcessCommand'' in a worker thread or process.
-''TryStartCommand()'' marks the command started and records the host/process identity so the same command is not executed by multiple HostDaemon instances.
-==== 7. HostDaemon creates the guest ====
-''product/website/HostDaemon/Handlers/CreateGuestCommandHandler.cs''
-''CreateGuestCommandHandler.Handle()'' performs the actual host-side build:
-  - blocks/reschedules if the invoice is not ready;
-  - connects to the guest's assigned host;
-  - checks banned-customer and disk-exists conditions;
-  - creates virtual disks;
-  - issues licences;
-  - installs the operating system;
-  - configures time machine if enabled;
-  - starts the guest;
-  - waits for ping/configures the OS;
-  - finalises successful creation.
-On successful creation, ''FlagGuestAsCreated()'' runs:
-<code>
-command.Guest.SetStatus(GuestStatus.Active);
-_guestService.Commit(command.Guest);
-</code>
-So the source-backed transition is ''Building'' to ''Active'', not ''Running''.
-==== 8. Command completion ====
-''product/website/HostDaemon/ProcessCommand.cs''
-After the handler returns, ''ProcessCommand.Run()'' calls:
-<code>
-command.SetCompleted();
-broker.Commit(command);
-handler.Completed();
-broker.Commit(command);
-</code>
-There is no separate HTTP callback from HostDaemon to the API in this path. HostDaemon updates the shared database state directly through NHibernate/brokers.
-===== Billing Behaviour =====
-There are two separate billing concerns:
-  * **Immediate API-side payment attempt:** ''GuestPlanApiService.CreateServer()'' calls ''InvoiceService.TryProcessInvoice()'' when the command's blocking invoice requires payment.
-  * **HostDaemon invoice gate:** ''BlockingHostCommandHandler'' checks ''command.BlockingInvoice()''. If the invoice is final and unpaid, HostDaemon does not start provisioning yet.
-''InvoiceService.TryProcessInvoice()'':
-  * refreshes the customer;
-  * returns true if no payment is required;
-  * marks the invoice ready for sending;
-  * handles zero-dollar invoices;
-  * uses customer credit first;
-  * generates a credit-card deposit if needed and possible;
-  * subtracts customer credit for debit invoices;
-  * calls ''invoice.MarkAsPaid()'' on success.
-HostDaemon does not appear to "finalise" the invoice during successful provisioning. The create command uses the invoice as a blocker. The API/payment path pays it before or while the command waits.
-For postpaid invoices, ''BlockingHostCommandHandler.BlockedOnInvoice()'' treats non-final invoices differently: it only blocks if the customer is not subscribed. ''InvoiceService.FinalizePostpaidInvoice()'' exists, but it is not part of the normal successful ''CreateGuestCommandHandler'' path shown here.
-===== Failure and Recovery Behaviour =====
-==== Unpaid invoice / pending payment ====
-''product/website/HostDaemon/BlockingHostCommandHandler.cs''
-If the blocking invoice is not paid:
-  * HostDaemon reschedules the command by default using ''RetryAfter()'' with a 1 minute wait.
-  * It cancels instead when the invoice is overdue, cancelled/final, or the command has waited at least 14 days.
-For create guest cancellation, ''CreateGuestCommandHandler.Cancel()'':
-  * ends the lease;
-  * cancels the invoice when needed;
-  * removes allocated resources from the guest;
-  * sets guest status to ''Terminated'';
-  * sends a guest-creation-cancelled email when appropriate.
-==== Exception during provisioning ====
-''CreateGuestCommandHandler.HandleException()'':
-  * logs a ''CreateGuestFailure'' system action;
-  * removes generated password result data;
-  * attempts unclean poweroff;
-  * attempts to remove the primary disk;
-  * cancels/refunds the invoice transaction if an invoice exists;
-  * ends the current lease;
-  * cancels the guest;
-  * sets guest status to ''Suspended'';
-  * revokes licences.
-The comment explains that failed creates used to delete the guest entirely, but now leave it suspended so normal automatic termination can clean it up.
-==== Failure while recording command state ====
-''product/website/HostDaemon/Hosted/FailedCommandQueue.cs''
-If command loading or committing fails, ''ProcessCommand'' enqueues a ''FailedCommand''. ''FailedCommandQueue'' later tries to load the command, run handler abort logic, call ''command.SetFailed()'', and commit that state. If the failure handler itself fails for non-database reasons, it logs and proceeds rather than retrying indefinitely.
-===== Data / State Transitions =====
-^ Object ^ Initial / queued state ^ Successful state ^ Failure state ^
-| ''Guest.Status'' | ''Building'' | ''Active'' | ''Suspended'' for provisioning exception; ''Terminated'' for invoice-cancel path |
-| ''HostCommand'' | not started / not completed | completed, succeeded | completed/failed where failure can be recorded |
-| ''Invoice'' | lease-term invoice, may require payment | paid by ''TryProcessInvoice()'' when payment succeeds | cancelled/refunded on provisioning failure or cancellation path |
-| ''Guest.Host'' | assigned before command execution | retained | resources may be removed/cancelled |
-| ''Guest.Interfaces'' | primary/additional assigned before command execution | retained | resources may be removed/cancelled |
-===== Support-Relevant Failure Points =====
-  * Customer has unpaid/overdue invoices.
-  * Customer has no verified payment details / is not subscribed.
-  * Customer is at guest limit.
-  * Plan is inactive or not listable.
-  * No host/network/IP is available.
-  * HostCommand is blocked by another active command.
-  * Payment cannot be processed.
-  * HostDaemon cannot create disk, install OS, start guest, or reach guest by ping.
-  * HostDaemon/DB commit fails and the failure queue needs to mark the command failed.
-===== Source Files =====
-  * ''product/website/WebApi/Services/GuestPlanApiService.cs''
-  * ''product/website/Domain/Services/GuestCreationService.cs''
-  * ''product/website/Domain/Services/GuestFactory.cs''
-  * ''product/website/Domain/Services/GuestProvisioningService.cs''
-  * ''product/website/Domain/Services/GuestManagementService.cs''
-  * ''product/website/Domain/Entities/CreateGuestCommand.cs''
-  * ''product/website/Domain/Services/HostCommandService.cs''
-  * ''product/website/Persistence/HostCommandBroker.cs''
-  * ''product/website/HostDaemon/Hosted/HostCommandThread.cs''
-  * ''product/website/HostDaemon/ProcessCommand.cs''
-  * ''product/website/HostDaemon/BlockingHostCommandHandler.cs''
-  * ''product/website/HostDaemon/Handlers/CreateGuestCommandHandler.cs''
-  * ''product/website/Domain/Services/InvoiceService.cs''
-===== Remaining Questions =====
-  * Which customer-visible status label maps to internal ''GuestStatus.Active'' in the current panel/API?
-  * Which operational job performs automatic cleanup/termination of suspended failed-create guests, and after what delay?
-  * Which logs/queries should support use first when a customer reports a server stuck in ''Building''?
-===== Related Wiki Pages =====
-  * [[compute-host-stack|Compute Host Stack]]
-  * [[compute-repo-structure|vps/compute Repo Structure]]
-  * [[infrastructure-services|Infrastructure Services]]
-  * [[wiki:jargon|Jargon / Reference Index]]