User Tools

Site Tools


syntheses:vps-create-flow

This is an old revision of the document!


VPS Creation / Provisioning Flow

This page traces the source-backed flow for creating a VPS from the customer/API request through to HostDaemon provisioning on the compute host.

Scope

  • Main repo: vps/vps project 1
  • Main application area: product/website
  • Compute execution: HostDaemon and host command handlers
  • Related repo: vps/compute project 22, for deployed compute-host programs and scripts

Short Version

  1. The API accepts a create-server request and validates the customer, plan, OS, network, billing state, and requested options.
  2. Domain services create a Guest in the database with status Building.
  3. Provisioning services assign the lease, host, network, primary interface/IPs, backup schedule, and default firewall state.
  4. GuestManagementService.CreateGuest() creates a CreateGuestCommand.
  5. HostDaemon polls the HostCommand table, marks the command started, and runs CreateGuestCommandHandler.
  6. On success, HostDaemon sets the guest status to Active, commits the guest, records system actions/host logs, sends mail, updates network members, and marks the command completed.
  7. On failure, HostDaemon attempts cleanup/refund/cancellation and leaves the guest suspended or terminated depending on where the failure occurred.

API and Domain Flow

1. API request

product/website/WebApi/Services/GuestPlanApiService.cs

GuestPlanApiService.CreateServer():

  • fetches the customer via CustomerResourceService.GetCustomer();
  • checks customer.Subscribed, except where signup flow allows otherwise;
  • validates purchase eligibility, guest limit, DTO fields, and active/listed plan;
  • resolves creation parameters such as facility, plan, operating system, network, and extras;
  • creates the guest/domain object inside a transaction;
  • flushes NHibernate so the API ID exists;
  • attempts immediate invoice processing when the blocking invoice requires payment;
  • returns an action DTO for the created host command.

Relevant source notes:

  • GuestPlanApiService.CreateServer() calls GuestCreationService.CreateNewGuestDomain() inside TransactionRunner.Transaction().
  • If command.BlockingInvoice().RequiresPayment(), it calls InvoiceService.TryProcessInvoice(blockingInvoice, null).

2. Guest/domain creation

product/website/Domain/Services/GuestCreationService.cs

GuestCreationService.CreateNewGuestDomain():

  • repeats purchase and guest-limit checks at domain level;
  • validates private IP/VPC details;
  • creates the guest entity through GuestFactory.InitializeNewGuest();
  • validates the guest;
  • applies options such as port blocks, IPv6, split NICs, SSH keys, script/user-data inputs, and duration;
  • calls GuestProvisioningService.ProvisionNewGuest();
  • commits the guest;
  • handles referral/fraud-review side effects;
  • queues the host command through GuestManagementService.CreateGuest().

3. Initial guest state

product/website/Domain/Services/GuestFactory.cs

GuestFactory.InitializeNewGuest() sets up the in-memory/new database guest and calls:

guest.SetStatus(GuestStatus.Building);

It also sets initial host/storage/CPU placeholders and applies the selected operating system and plan.

4. Provisioning assignments

product/website/Domain/Services/GuestProvisioningService.cs

ProvisionNewGuest() performs the pre-daemon allocations:

  1. GuestLeaseCreationService.CreateNewLeaseForGuest() creates the lease and lease term.
  2. HostService.AddGuestToAvailableHost() assigns a compute host.
  3. NetworkService.AddGuestToNetwork() assigns a network.
  4. GuestService.AssignPrimaryInterfaceToGuest() assigns the primary interface/IP and commits the guest.
  5. GuestService.AssignAdditionalInterfaces() handles extra interfaces/VPC NICs.
  6. BackupScheduleService.SetupGuestBackupSchedule() sets backup schedule state.
  7. SetupDefaultFirewall() clears/initialises firewall rules.

HostCommand / HostDaemon Flow

5. CreateGuestCommand is queued

product/website/Domain/Services/GuestManagementService.cs

CreateGuest() creates a CreateGuestCommand, attaches password/SSH/script/user-data inputs, and calls HostCommandService.AddHostCommand().

product/website/Domain/Entities/CreateGuestCommand.cs

The command has six visible steps:

  1. Requesting server resources
  2. Allocating virtual disk
  3. Installing operating system
  4. Starting server
  5. Configuring operating system
  6. Processing invoice

It implements IInvoiceBlockableCommand. Its blocking invoice is:

Guest.LatestLeaseTerm.Invoice

6. HostDaemon picks up the command

product/website/HostDaemon/Hosted/HostCommandThread.cs

HostDaemon repeatedly:

  1. asks HostCommandBroker.GetNextCommand() for the next not-started, not-completed, unblocked command;
  2. calls HostCommandService.TryStartCommand();
  3. starts ProcessCommand in a worker thread or process.

TryStartCommand() marks the command started and records the host/process identity so the same command is not executed by multiple HostDaemon instances.

7. HostDaemon creates the guest

product/website/HostDaemon/Handlers/CreateGuestCommandHandler.cs

CreateGuestCommandHandler.Handle() performs the actual host-side build:

  1. blocks/reschedules if the invoice is not ready;
  2. connects to the guest's assigned host;
  3. checks banned-customer and disk-exists conditions;
  4. creates virtual disks;
  5. issues licences;
  6. installs the operating system;
  7. configures time machine if enabled;
  8. starts the guest;
  9. waits for ping/configures the OS;
  10. finalises successful creation.

On successful creation, FlagGuestAsCreated() runs:

command.Guest.SetStatus(GuestStatus.Active);
_guestService.Commit(command.Guest);

So the source-backed transition is Building to Active, not Running.

8. Command completion

product/website/HostDaemon/ProcessCommand.cs

After the handler returns, ProcessCommand.Run() calls:

command.SetCompleted();
broker.Commit(command);
handler.Completed();
broker.Commit(command);

There is no separate HTTP callback from HostDaemon to the API in this path. HostDaemon updates the shared database state directly through NHibernate/brokers.

Billing Behaviour

There are two separate billing concerns:

  • Immediate API-side payment attempt: GuestPlanApiService.CreateServer() calls InvoiceService.TryProcessInvoice() when the command's blocking invoice requires payment.
  • HostDaemon invoice gate: BlockingHostCommandHandler checks command.BlockingInvoice(). If the invoice is final and unpaid, HostDaemon does not start provisioning yet.

InvoiceService.TryProcessInvoice():

  • refreshes the customer;
  • returns true if no payment is required;
  • marks the invoice ready for sending;
  • handles zero-dollar invoices;
  • uses customer credit first;
  • generates a credit-card deposit if needed and possible;
  • subtracts customer credit for debit invoices;
  • calls invoice.MarkAsPaid() on success.

HostDaemon does not appear to “finalise” the invoice during successful provisioning. The create command uses the invoice as a blocker. The API/payment path pays it before or while the command waits.

For postpaid invoices, BlockingHostCommandHandler.BlockedOnInvoice() treats non-final invoices differently: it only blocks if the customer is not subscribed. InvoiceService.FinalizePostpaidInvoice() exists, but it is not part of the normal successful CreateGuestCommandHandler path shown here.

Failure and Recovery Behaviour

Unpaid invoice / pending payment

product/website/HostDaemon/BlockingHostCommandHandler.cs

If the blocking invoice is not paid:

  • HostDaemon reschedules the command by default using RetryAfter() with a 1 minute wait.
  • It cancels instead when the invoice is overdue, cancelled/final, or the command has waited at least 14 days.

For create guest cancellation, CreateGuestCommandHandler.Cancel():

  • ends the lease;
  • cancels the invoice when needed;
  • removes allocated resources from the guest;
  • sets guest status to Terminated;
  • sends a guest-creation-cancelled email when appropriate.

Exception during provisioning

CreateGuestCommandHandler.HandleException():

  • logs a CreateGuestFailure system action;
  • removes generated password result data;
  • attempts unclean poweroff;
  • attempts to remove the primary disk;
  • cancels/refunds the invoice transaction if an invoice exists;
  • ends the current lease;
  • cancels the guest;
  • sets guest status to Suspended;
  • revokes licences.

The comment explains that failed creates used to delete the guest entirely, but now leave it suspended so normal automatic termination can clean it up.

Failure while recording command state

product/website/HostDaemon/Hosted/FailedCommandQueue.cs

If command loading or committing fails, ProcessCommand enqueues a FailedCommand. FailedCommandQueue later tries to load the command, run handler abort logic, call command.SetFailed(), and commit that state. If the failure handler itself fails for non-database reasons, it logs and proceeds rather than retrying indefinitely.

Data / State Transitions

Object Initial / queued state Successful state Failure state
Guest.Status Building Active Suspended for provisioning exception; Terminated for invoice-cancel path
HostCommand not started / not completed completed, succeeded completed/failed where failure can be recorded
Invoice lease-term invoice, may require payment paid by TryProcessInvoice() when payment succeeds cancelled/refunded on provisioning failure or cancellation path
Guest.Host assigned before command execution retained resources may be removed/cancelled
Guest.Interfaces primary/additional assigned before command execution retained resources may be removed/cancelled

Support-Relevant Failure Points

  • Customer has unpaid/overdue invoices.
  • Customer has no verified payment details / is not subscribed.
  • Customer is at guest limit.
  • Plan is inactive or not listable.
  • No host/network/IP is available.
  • HostCommand is blocked by another active command.
  • Payment cannot be processed.
  • HostDaemon cannot create disk, install OS, start guest, or reach guest by ping.
  • HostDaemon/DB commit fails and the failure queue needs to mark the command failed.

Source Files

  • product/website/WebApi/Services/GuestPlanApiService.cs
  • product/website/Domain/Services/GuestCreationService.cs
  • product/website/Domain/Services/GuestFactory.cs
  • product/website/Domain/Services/GuestProvisioningService.cs
  • product/website/Domain/Services/GuestManagementService.cs
  • product/website/Domain/Entities/CreateGuestCommand.cs
  • product/website/Domain/Services/HostCommandService.cs
  • product/website/Persistence/HostCommandBroker.cs
  • product/website/HostDaemon/Hosted/HostCommandThread.cs
  • product/website/HostDaemon/ProcessCommand.cs
  • product/website/HostDaemon/BlockingHostCommandHandler.cs
  • product/website/HostDaemon/Handlers/CreateGuestCommandHandler.cs
  • product/website/Domain/Services/InvoiceService.cs

Remaining Questions

  • Which customer-visible status label maps to internal GuestStatus.Active in the current panel/API?
  • Which operational job performs automatic cleanup/termination of suspended failed-create guests, and after what delay?
  • Which logs/queries should support use first when a customer reports a server stuck in Building?
syntheses/vps-create-flow.1784258136.txt.gz · Last modified: by 127.0.0.1

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki