3 posts tagged with "Proxmox"

Version v3.1.5 closes out the 3.1 line. It adds no new product surface - it makes the surface added in v3.1.0 and v3.1.2 behave consistently. The admin REST API now matches the panel and its own documentation, load balancer routing and Proxmox firewall synchronization are more dependable, alert mail is delivered on its own queue, and every instance operation is fully localized. A platform review ran alongside this work; its outcomes are folded into the items below.

Upgrading is routine. There is nothing to reconfigure and no behavior to relearn.

• [Improvement] Admin REST API consistency - updating a user no longer requires resending a password, the acting administrator resolves the same way on every endpoint, and account credentials are excluded from responses. Load balancer security group rules are readable from the administrative API.
• [Improvement] Load balancer routing rules honor catch-all matches, applied after the specific rules and before the default backend, so precedence follows the order the rule list reads.
• [Improvement] Proxmox security group synchronization converges reliably, so firewall rule changes reach running VMs on every cycle, and an explicit deny takes precedence over a broader allow.
• [Improvement] Notification and alert mail - certificate lifecycle notices, autoscaler token renewals, the admin income digest, and invoice issuance now use the dedicated notifications queue rather than sharing one with live dashboard traffic.
• [Improvement] Full localization of instance operations, with a build-time guard so a new action cannot ship without its wording.
• [Improvement] Quota enforcement is identical from the panel and the API, and quota messages distinguish being at your limit from a brief collision with your own concurrent request.
• [Security] Tighter tenant boundaries on object storage keys and volume plans, stricter identity binding for cluster nodes, and payment capture bound to its originating transaction.
• [Security] Cluster join credentials are cleared on every terminal outcome, with kubernetes:scrub-cloudcfg provided for existing installs.
• [Operations] Managed MariaDB monitoring, scheduler health naming, and a single pinned PHP runtime for workers and cron.

Since v3.1.0​

If you are upgrading from the 3.0 line, the two releases between it and this one carry the feature work:

• v3.1.0 adds native Proxmox VE support. Point the platform at an existing PVE 8 or 9 node or cluster with a single API token and manage it beside your KVM fleet - deploys, VPC networking, security groups, backups, snapshots, live migration, consoles, HA, Docker, managed databases, load balancers, Kubernetes, and exact per-NIC bandwidth metering. Agentless, over the Proxmox API.
• v3.1.2 adds OAuth sign-in with Google, Microsoft and GitHub, enforceable admin OIDC single sign-on, private locations with a built-in request-access workflow, a per-node deployment readiness engine surfaced on the dashboard, the hypervisor list and each node's detail page, and a substantially tougher load balancer with static-IP deploys, captured error state, and long-lived TCP session support.

v3.1.5 builds directly on both: the Proxmox firewall and load balancer improvements below apply to the surfaces those releases introduced.

Kubernetes cluster credential hygiene​

Cluster join credentials are meant to be short-lived - they exist while a node is joining and are cleared once it settles. That clearing now runs on every terminal outcome, so a node that fails to join is cleaned up exactly like one that succeeds, and it runs from the periodic reconcilers as well as the join callback, covering a node that never reports back.

Because the behavior changed, v3.1.5 ships php artisan kubernetes:scrub-cloudcfg for existing installs. It is a dry run by default and requires an explicit --force to write, only touches clusters that are fully destroyed, holds a settle window on top of that, includes soft-deleted records, and logs everything it does. Operators upgrading from an earlier 3.1.x can run it once; new installs never accumulate this data.

Admin REST API​

Several /api/v1 endpoints behaved differently from the panel and from their own documentation. They now agree:

• Updating a user no longer requires a password. You can change a name, a role, or a quota without sending a credential, and an unchanged email address is accepted.
• The acting administrator resolves consistently across instance image listing, user update and delete, VPC enable and disable, and backup creation.
• Account secrets never appear in responses - API credentials and multi-factor state are excluded from every serialized user object, and the published examples match.

The API manifest, the OpenTofu provider, and the MCP server coverage gates remain in lockstep at 889 endpoints, verified in CI.

Tenant boundaries​

Object storage access keys resolve strictly within the owning account on every user-facing route, while the administrative surface keeps its intentionally cross-account view. Volume plans are enforced against the region that offers them, so a plan wired to one location cannot be provisioned into another and pricing stays aligned with your catalog. Cluster node identity is derived solely from the signed token a node presents when it joins.

None of this requires a schema change or any action on your part.

Localization​

Every user-facing string in the instance operation pipeline is now translatable, including task names and result messages assembled at runtime. A build-time guard reads the accepted actions directly from the service and fails the build if any of them lacks its wording, so the coverage cannot silently regress.

Version v3.1.2 is the trust and hardening release that follows the Proxmox debut in v3.1.0. It modernizes how people get into the panel - social sign-in for users and enforceable OIDC single sign-on for admins - and how you sell capacity, with private locations and a built-in request-access workflow. Operators get a per-node deployment readiness engine and a substantially tougher load balancer. Underneath, this cycle ran two full platform security sweeps plus dedicated audits of the backup system and managed databases, on both the master and the hypervisor agent.

• [Feature] Sign in with Google, Microsoft, or GitHub - users can register and log in through OAuth, link and unlink providers from their profile, and auto-link to an existing account only when the provider asserts a verified email.
• [Feature] Admin single sign-on (OIDC) - bind admin logins to your identity provider with strict subject binding and no just-in-time provisioning, optionally enforce SSO for all admin password logins, and keep a time-limited break-glass path for IdP outages.
• [Feature] Private locations with request access - lock any location to selected accounts. Locked regions stay visible in the catalog with a lock treatment, users request access in one click, and admins approve or deny from a dedicated queue with email notifications both ways.
• [Feature] Node deployment readiness - every hypervisor now carries a live readiness checklist (agent, storage, network, capacity, deploy gates) surfaced as a dashboard card, a fleet list badge, and a per-node checklist with failure-specific fix hints.
• [Feature] Load balancers on allocated static IPs, captured error state (full detail for admins, a subtle banner for users), and per-frontend idle timeouts - TCP frontends now default to one-hour timeouts with kernel keepalives, so SSH and database sessions through an LB no longer drop at 50 seconds.
• [Feature] Proxmox surface expansion - VM snapshots, instance tags, guest-agent IP discovery, and backup file-restore in the user API; node issues, scheduled backup jobs, and live migration in the admin API; and admin edits to resources, topology, boot order, and NICs now push live to running VMs.
• [Feature] Security group drop rules on KVM - rule actions are honored end to end, so explicit drop rules override broader accepts, matching the Proxmox behavior.
• [Improvement] Teams - instance password mails go to the account owner with every instance-manage member in CC. Admin task queue gains one-click pruning and clean deletion.
• [Security] Two platform-wide security sweeps, defense-in-depth guardrails for the AI assistant, a backup-system audit in three phases, and a managed-database hardening batch. Details below.

Sign in with Google, Microsoft, and GitHub​

The login and registration pages now offer OAuth sign-in for Google, Microsoft, and GitHub. Each provider is enabled individually in the admin settings with its own client credentials; nothing shows on the login page until a provider is configured and switched on.

The linking rules are deliberately conservative, because OAuth auto-linking is a classic account-takeover vector:

• An OAuth identity auto-links to an existing account only when the provider asserts the email as verified. Google must present a true email_verified claim, GitHub only ever returns primary-and-verified addresses, and Microsoft sign-ins are validated against the tenant-verified UPN with the known cross-tenant takeover patterns (nOAuth) explicitly rejected.
• Sign-ups that arrive without a usable verified email go through a complete-profile step instead of silently creating a half-formed account, and the account write is transactional so a double submit cannot orphan a user.
• Logged-in users manage linked providers from their profile: connect, view, and unlink, with relinking handled safely.

Admin single sign-on (OIDC)​

Admin access can now be delegated to your identity provider - Okta, Entra ID, Keycloak, or any OIDC-compliant IdP:

• Strict binding. An admin's IdP identity binds on sub (subject), never on mutable claims, and there is no just-in-time provisioning - only pre-existing admin accounts can bind. The first bind is forensically logged, and stale identities are deleted rather than left dangling.
• Enforcement. Once your IdP is verified, you can require SSO for all admin password logins. The enforcement policy carries a lockout interlock so you cannot switch it on in a state that would lock every admin out.
• Break-glass. For IdP outages, php artisan admin:sso-break-glass opens a time-limited bypass that expires on its own. It is a deliberate, logged, console-only action.
• Setup UI. A new Authentication settings tab covers both features, including an OIDC discovery test that validates your issuer before anything is enforced. HTTPS is required and JWT verification is always on.

Private locations and request access​

Locations (hypervisor groups) can now be restricted per account. The catalog stays honest about what exists:

• Locked regions render on every create surface - the deploy modal, Cloud Service, self-provisioning, VPC and Kubernetes pickers - with a frosted lock treatment and the region name still visible, instead of vanishing from the catalog.
• Users hit Request access on a locked location, confirm, and the request lands in a new admin queue with a navigation badge. Admins approve or deny inline; both outcomes notify the user by mail. Access states are tracked per account as available, requested, or locked.
• Admin user pages gain a Cloud Service tab consolidating the account's location grants, inline approve and deny, and the account's provisioning limits.
• A per-account cloud provisioning switch cleanly disables self-service provisioning for an account without touching its running services, and the billing-exemption logic was made consistent across every surface that renders a deploy button.

Access enforcement is server-side on every create path, across web, API, queue, and AI-assistant surfaces. The lock UI is presentation; the gate is in the services.

Node deployment readiness​

Answering "why is nothing deploying to this node" used to mean reading logs. Now every hypervisor - KVM and Proxmox - carries a readiness engine that evaluates the conditions a deploy actually requires: agent reachability, storage presence and free capacity, subnet availability, deploy flags, maintenance and lock state.

• The admin dashboard shows a fleet readiness card.
• The hypervisor list badges each node ready, pending, or blocked.
• The node detail page renders the full checklist, and every failed check carries a specific fix hint tied to the actual failure, not a generic message.
• Adding a Proxmox node now runs its first cluster reconcile synchronously, so a freshly linked node reports honest readiness immediately instead of waiting for the next cron pass.

The checks mirror the real deploy gates - a node the checklist calls ready is a node the scheduler will actually use.

Load balancer improvements​

• Static IP deploys. User load balancers can deploy onto allocated static IPs, so an LB's address can be planned, firewalled, and DNS'd before it exists.
• Error surfacing. LB provisioning and sync failures are captured as a last-error state: admins see the full detail on the LB page, users see a subtle banner that something is being worked on - operational detail stays internal.
• Long-lived TCP sessions. TCP-mode frontends previously inherited HTTP-tuned 50-second idle timeouts, which silently killed idle SSH, database, and message-queue connections through the LB. TCP frontends and backends now default to one-hour timeouts with kernel TCP keepalives on both sides, websocket tunnels get a matching post-upgrade timeout, and every frontend gains an optional idle timeout field (30 to 86400 seconds) in both the user and admin panels for workloads that need more or less.
• Kubernetes LB fixes. Service LBs honor the managed-loadbalancer-public-ip annotation, weighted routing-rule backends materialize correctly with collision-free ACL names, port 80 stays plaintext under global SSL mode, and NodePort backends are health-checked over TCP.
• Plan enforcement. Standalone LB deploys enforce the location's plan-group offering, closing a path where an LB could deploy from a plan the region does not sell.

Proxmox, continued​

v3.1.0 shipped the driver; v3.1.2 finishes the surfaces around it:

• User API: VM snapshots (list, create with optional RAM state, rollback, delete), instance tags, guest-agent IP discovery, and backup file-restore browse and download.
• Admin API: node issues (list, retry, resolve), PVE scheduled backup jobs, and live migration with precheck. Route binder failures return real 404s instead of leaking existence.
• Live VM edits. Admin changes to resources, CPU topology, boot order, and NICs push to the running VM where PVE allows it, with CPU flags, secure boot, and TPM handling brought to parity with KVM.

All new endpoints are covered by the API manifest and mirrored in the OpenTofu provider and MCP server coverage gates.

Reliability: Kubernetes, VPN gateways, VPC​

• Kubernetes: worker-pool scale-up crash fixed, long jobs no longer double-execute after 90-second queue redelivery, control-plane and worker plan pickers are scoped to the region's plan groups, node selection prefers the NAT-active hypervisor, and deploys survive recycled-IP ARP staleness and transient agent transport blips. Control-plane LB deploys from the queue were failing on an authentication-context assumption; provisioning gates now evaluate the acting user everywhere.
• VPN gateways: peer key pairs auto-generate as the UI always promised, and road-warrior clients receive the VPC's private DNS resolver.
• VPC on KVM: cross-node NAT egress now installs the correct default route on non-active nodes and repairs it in the periodic sync, the VPC bridge joins a firewalld zone so nftables cannot silently reject its traffic, and ICMP redirects are suppressed on VPC veths - closing a class of "works from one node, dead from another" reports.
• Node provisioning: fresh hypervisors install required CLIs rather than only upgrading existing ones, Debian contrib is enabled across both source layouts for ZFS, and half-merged /usr systems are repaired so kernel modules and ufw work on broken base images.

Managed database hardening​

The managed database service went through a dedicated audit. Highlights: six critical backup, restore, and HA defects fixed; incremental backup chain source pinning so a restore can never mix chains; encryption keys moved off process argv; a watchdog that rescues clusters stuck in configuring with init-phase visibility; callback token lifecycle hardening with a localhost guard; PostgreSQL cluster self-heal; replica resync credentials forwarded correctly; and the admin password revealed on the detail pages where operators actually need it.

Backup system audit​

A three-phase audit of the backup pipeline shipped on both sides:

• Master: failure alerting is throttled and queue-routed so it always sends, repeated failures auto-pause a plan instead of burning nightly cycles, prune notifications report what was actually pruned, backup sizes are captured from the agent callback, and remote restores gained a direct download path while a dead restore path was removed.
• Agent: a credential leak into backup artifacts was stopped, silently truncated backups are now detected and failed, backup and restore state files are no longer world-readable, and qcow restores verify the staged artifact and use tmp-then-rename so a partial download can never replace a disk.

Security sweeps​

Two platform-wide sweeps (2026-07-29 and 2026-07-30) ran during this cycle, with every finding remediated before release. The notable classes:

• Billing integrity: top-up capture is now bound to its originating transaction, closing a credit-fraud path; credit adds are validated; backup debits are atomic.
• Tenant scoping: SSH sessions, S3 access keys, Kubernetes certificate renewal, and VPC selection are all bound to the owning tenant; state-changing restore moved off GET.
• Auth: password-reset throttling, no exception reflection to clients, OAuth and email uniqueness guarantees, and the Microsoft cross-tenant (nOAuth) rejections described above.
• Secrets at rest and in transit: queue payloads carrying secrets are encrypted, failed-job rows are pruned, Kubernetes join credentials no longer travel through cloud-init user data, notification channel secrets are no longer serialized into events, and WireGuard AllowedIPs are validated before any privileged guest execution.
• AI assistant guardrails (three phases of defense in depth): streaming egress redaction of configured secrets, knowledge-base audience scoping that fails closed, untrusted-data framing around tool output with prompt-injection guards, and redaction of persisted tool calls and audit logs so the assistant's own storage cannot become the leak.
• Dependencies: dompdf bumped for CVE-2026-56722.

Version v3.1.0 is the Proxmox release. The platform now speaks Proxmox VE natively: point it at any existing PVE 8 or 9 node or cluster with a single API token, and every node is discovered, linked, and managed from the same panel, API, and billing pipeline as your KVM fleet. There is nothing to install inside Proxmox and no agent to maintain - the driver works entirely over the Proxmox REST API, with one optional one-command bootstrap on a node for the features that need host access (VPC WebSSH and exact per-NIC metering). Nothing changes for existing KVM hypervisors, instances, plans, or balances.

• [Feature] Native Proxmox VE driver - full instance lifecycle on PVE 8.0+ and 9.x: deploy, power, suspend and resume, reinstall, resize, destroy, ISO mounting, cloud-init, GPU passthrough, VM tags, and per-tenant resource pools. Agentless by design; see the section below.
• [Feature] One-token cluster onboarding - give the panel one reachable member IP and one API token, and it discovers every node in the cluster, links them all, verifies SDN and FRR prerequisites, provisions console credentials, and self-checks the token's privileges. Failures are listed per node instead of aborting the lot.
• [Feature] VPC networking on Proxmox SDN - VPCs are realized as PVE SDN EVPN zones, vnets, and subnets, with NAT gateways, hot attach and detach of VPC NICs, and automatic reconciliation. Security groups translate to the PVE firewall with accept and drop rules, IP sets, and per-VM rulesets.
• [Feature] Backups and snapshots - vzdump backup, restore, and delete per instance, optional live-restore, native PVE scheduled backup jobs with full retention control, file-level restore (browse a backup and download individual files), and VM snapshots with optional RAM state, rollback, and delete.
• [Feature] Live migration - move a running VM between cluster nodes from the panel, with an eligibility precheck, local-disk handling, target storage selection, bandwidth limits, and a dedicated migration network option.
• [Feature] High availability via the PVE CRM - instances enroll into Proxmox's own HA stack on deploy, with per-group tuning and placement rules; the panel mirrors CRM state instead of running a second watchdog.
• [Feature] Full guest-service parity - Docker deployments, managed databases (including S3 backups, point-in-time recovery, and replication), load balancers with Let's Encrypt, Kubernetes clusters, WebSSH, SSH key injection, and password resets all work on Proxmox instances through the QEMU guest agent.
• [Feature] Exact bandwidth metering - per-NIC counters are read from the host, so VPC (east-west) traffic and public traffic are billed separately and never double-counted, with reset-safe accumulation across reboots.
• [Feature] Node Issues dashboard - operational failures on a Proxmox node (metering, host access, proxy installs, firewall gaps) surface as first-class admin issues that reopen if they recur, instead of hiding in a log file.
• [Improvement] Fleet-scale collection - metrics, statistics, security-group reconciliation, and HA reconciliation all fan out per node onto a worker pool, so a large cluster is collected in parallel rather than serially.

Native Proxmox VE support​

v3.1.0 introduces a second hypervisor backend alongside KVM. A hypervisor group is now either a KVM group or a Proxmox cluster, and both kinds run side by side on one panel with the same instances, plans, billing, and user experience.

The design principle is API-only: the driver talks to the Proxmox REST API and nothing else. Your cluster keeps looking like a normal Proxmox cluster - VMs created by the panel are plain QEMU VMs you can see in the PVE web UI, tasks the panel starts are ordinary PVE tasks (UPIDs the panel tracks and can cancel), and nothing on the node is patched or replaced.

Onboarding an existing cluster​

Create a hypervisor group of type Proxmox, paste one member's address and one API token, and the panel calls the cluster status endpoint to discover every node. Each node is linked individually, storage is discovered per node with its real PVE plugin type, an @pve console credential is provisioned for VNC, and SDN plus FRR prerequisites are verified so VPC problems are caught at link time rather than at first deploy. The token itself is self-checked: the panel reports the PVE version and the privileges the token actually holds, and flags the guest-agent privilege gap that PVE 9 introduced, right on the node's detail page.

Supported: Proxmox VE 8.0 and later, including 9.x, standalone nodes and clusters. Group names must match the PVE cluster name so the group is an unambiguous mirror of the datacenter, and groups are kept homogeneous - one group is either KVM or Proxmox, never a mix.

Images and deploys​

Operating system images are built once per cluster into a vzdump archive, then every deploy restores that archive into the customer's VM ID. There are no template VMs parked on your nodes and no full-clone storms: what lands is a plain VM, on the storage you chose, with cloud-init applied. Archives are labeled with the upstream image name, can be purged with one command, and image builds are excluded from orphan detection.

Cloud-init works two ways: the default NoCloud seed ISO (full feature set, including multi-user and multi-IP layouts), or PVE's native cloud-init as a per-group opt-in for simple single-user, single-IP instances where a config drive is unnecessary.

Existing VMs on the cluster are not stranded either: the orphan importer can adopt any VM the panel did not create - including raw, ZFS, LVM, and Ceph-backed disks - and attach it to a user as a managed instance.

Storage​

All common PVE storage backends are supported and detected with their real capabilities: directory, NFS, LVM, LVM-thin, ZFS, and Ceph RBD, including onboarding an external Ceph cluster directly from the panel. Extra volumes allocate through the PVE storage API with the correct format per backend, hot-plug with IO limits, and resize live (grow-only, as QEMU requires). Capability gates are honest: operations a given backend cannot do, such as snapshots on plain LVM, are refused with a clear message instead of failing halfway.

Networking, VPCs, and security groups​

VPCs on a Proxmox group are built on PVE SDN: an EVPN controller and zone per VPC with a dedicated VRF VNI, a vnet and subnet per VPC subnet, and a NAT gateway with SNAT semantics that hold cluster-wide. VPC NICs hot attach and detach on running VMs. A reconciler keeps the SDN objects healthy and converges NAT gateway activation, and VPC create and destroy push eagerly so the fabric follows the panel immediately.

Security groups compile to the PVE firewall: cluster-level groups and IP sets, per-VM rulesets with both accept and drop actions, anti-spoofing IP filters, and firewall flags only on the NICs they belong to. Reconciliation is fingerprint-based, so unchanged nodes are skipped, and a cron backstop repairs drift. The datacenter firewall is enabled safely, preserving management access rather than default-dropping the hosts.

The instance network tab also gains guest IP discovery on Proxmox: addresses are read live from the QEMU guest agent, filtered of loopback and link-local noise, for both admins and users.

Consoles and WebSSH​

Three console paths ship for Proxmox instances:

• VNC (noVNC) through a PVE login ticket with single-use, server-side session tokens. PVE credentials and tickets never reach the browser.
• Serial terminal (xterm.js over termproxy), tenant-scoped.
• WebSSH - a real SSH shell in the browser. Public-IP guests are dialed directly. VPC-only guests are reached through proxmox-ssh-proxy, a small node-side service that enters the VPC's VRF on the host, because the guest's route only exists there.

The proxy is the one place host access is needed, and it bootstraps with one command on one node: the panel mints a short-lived, single-use install command, the script fetches the binary from your panel (never a third-party artifact), installs the service, opens the master's port in the PVE firewall using the egress address the node itself observed, and the panel keeps every node's proxy current automatically from then on. Install state is verified against what is actually on the node, so a half-finished install shows up as fixable instead of hiding.

Backups, scheduled jobs, and file restore​

Per-instance backups run through vzdump with completion tracked to the panel's backup queue, restore (optionally live-restore, so the VM boots while data streams back), redirect-to-storage on restore, and delete. Cluster-level scheduled backup jobs are managed from the panel with the full keep-last, keep-hourly, keep-daily, keep-weekly, keep-monthly, and keep-yearly retention set, targeting specific VMs, a resource pool, or the whole cluster. With Proxmox Backup Server storage, file-level restore lets admins and users browse a backup's filesystem and download individual files without restoring the VM.

Snapshots and Forge​

VM snapshots are first-class on capable storage: create with or without RAM state, list, rollback, and delete, from both admin and user instance pages. Forge (the checkpoint-try-rollback workflow) rides the same mechanism with a reserved snapshot slot, protected from name collisions with user snapshots.

Live migration​

Admins can live-migrate a Proxmox instance between cluster nodes with a precheck that refuses ineligible targets and local hardware blockers, handles local disks, and passes through target storage, bandwidth limit, migration network, and online mode. On PVE 9, conntrack-state migration is available as an opt-in. Migration success updates the instance's node assignment and flushes routing caches; a failed migration leaves the instance exactly where it was.

High availability​

Proxmox groups delegate HA to the PVE cluster resource manager - the layer that actually owns fencing and recovery - instead of duplicating it. Instances enroll on deploy and unenroll on destroy, groups expose HA tuning (and node-affinity placement, using HA groups on PVE 8 and node-affinity rules on PVE 9), and a reconciler mirrors CRM state back into the panel so the instance page shows the real HA state. The KVM HA watchdog is untouched, and a Proxmox-side failure can never stall KVM monitoring.

Guest services: Docker, databases, load balancers, Kubernetes​

Everything that previously required the slave agent's SSH path now runs over the QEMU guest agent on Proxmox instances:

• Docker: deploy, control, status, and logs.
• Managed databases: the full suite - configure, restart, password reset, backup tooling, full and incremental backups to S3, point-in-time recovery, restore, upgrade, replication, and batched health checks.
• Load balancers: configuration, Let's Encrypt issue and renew, and telemetry setup.
• Kubernetes: kubectl execution, control-plane certificate upload and renewal, and admin kubeconfig reissue, routed through the Proxmox adapter for Proxmox-hosted clusters.
• Instance basics: SSH key inject and remove with port detection, password resets, and WireGuard configuration sync.

Large payloads are delivered safely: small files write through the agent directly, larger ones are pulled by the guest from a single-use, checksum-verified URL.

Metering, billing, and scale​

Metrics and statistics feed the same billing pipeline as KVM, with the units and counters aligned so plan enforcement and charts behave identically. Two things are worth calling out:

• Exact per-NIC bandwidth. The PVE API only exposes aggregate VM counters, which cannot split public from VPC traffic. v3.1.0 reads per-NIC tap counters from the host instead, so VPC traffic and public traffic are metered separately and never double-billed, with reset-safe accumulation that keeps mid-day reboots billable.
• Parallel collection. Metrics, instance statistics, HA reconciliation, and security-group reconciliation fan out per node and per group onto a dedicated worker pool. Collection time scales with your largest node, not your node count.

Operational visibility​

A new Node Issues dashboard (admin, under Proxmox) surfaces operational failures on a node as structured issues: what failed, on which node, how many times, first and last seen. Issues reopen automatically if the condition recurs after being marked solved, and each issue carries a retry action. Long-running PVE tasks are tracked by UPID and can be cancelled from the panel, and a stuck panel task cancels its PVE counterpart when marked failed.

Security hardening in this release​

The Proxmox driver went through dedicated security review during the cycle. Notable fixes shipping in v3.1.0: a WireGuard configuration path that could execute as root in the guest was closed, WebSSH sessions no longer expose PVE authentication cookies, console ACLs are revoked rather than accruing, ISO URL fetching is SSRF-hardened, console routes are tenant-scoped with negative-authorization tests, and backup restore and file-restore endpoints verify the backup belongs to the instance before acting.