Files
harvester-ui-extension/pkg/harvester/store/harvester-store/getters.ts
T
Marcelo Fukumoto 0c6eee1744 feat: page the VM list on the server when the API supports it (#1261)
* feat: page the VM list on the server when the API supports it

On clusters whose API serves lists from the Steve SQL cache, the VM list
uses PaginatedResourceTable: it loads one page of VMs, and the VMIs, pods,
PVCs, PVs, migrations, restores and backups of those VMs only. Sorting,
search and the namespace filter run on the server, so CPU, memory, IP and
node can't be sorted or searched there.

The feature is gated by the vmListPagination feature flag and enabled for
the VM list only, through a pagination context. Older clusters keep the
current list.

Signed-off-by: Marcelo Fukumoto <marcelo.fukumoto@suse.com>

* feat: enable the VM list pagination from v1.10.0

The backend change that serves API lists from the SQL cache has to be on
the v1.10 branch before this is merged.

Signed-off-by: Marcelo Fukumoto <marcelo.fukumoto@suse.com>

* fix: keep the paged VM list when adding a volume

The Add Volume dialog loaded every VM to find the volumes in use, which
replaced the page the VM list keeps in the store and emptied the list.
When the list is paged, the dialog now reads only the VMs of the
namespace and doesn't store them.

The requests for the resources of a page use one IN filter instead of
one filter per value, which keeps the URLs about a third as long.

Signed-off-by: Marcelo Fukumoto <marcelo.fukumoto@suse.com>

* fix: keep the paged VM list intact in its dialogs and on older APIs

The Add Volume, Storage Migration and Restore dialogs read the VMs, PVCs
and backups of the VM's namespace through the VM. While the list keeps a
page of a type in the store they read the namespace without storing it,
so the list keeps its page and the dialogs' own lists don't shrink when
the page reloads. Storage Migration no longer offers volumes of VMs on
other pages as targets.

The list is paged only when the API lists the VM's printer columns as
metadata.fields, which the SQL cache does, so a missing or non-semver
server version can't turn it on against an API that can't filter.

One failed request for a page's resources no longer stops the others.

Signed-off-by: Marcelo Fukumoto <marcelo.fukumoto@suse.com>

* fix: show the restart notice only when its VMs change

The list rebuilds the names of the VMs that need a restart on every
refresh, and when it pages also on every page, sort and search, so the
notice popped up again each time even when nothing new needed a
restart. Show it only when the set of VMs changes.

While paging it still covers only the VMs on the current page: the API
can't filter VMs by their RestartRequired condition.

Signed-off-by: Marcelo Fukumoto <marcelo.fukumoto@suse.com>

* fix: find the launcher pods of every VM on the page

The paged list loaded the page's launcher pods by the harvesterhci.io/vmName
label, which only the Harvester VM form writes into a VM's template. VMs
created any other way, like an imported VM or one applied with kubectl,
got no pod, so their rows lost View Logs and the starting and pod error
states.

Load the pods after the page's VMIs instead, by the kubevirt.io/created-by
label KubeVirt puts on every launcher pod (the VMI's UID).

Also cover the paged mode in the tests: it must not load every VMI,
migration or pod, and a forced refresh must reach every request.

Signed-off-by: Marcelo Fukumoto <marcelo.fukumoto@suse.com>

* fix: notify about restarts again when coming back to the VM list

The restart notice compared the VMs that need a restart with the
previous value of the list. Coming back from a page that loads every VM,
like Volumes or a VM's detail page, that previous value already held the
same VMs, so the notice didn't show. Remember what was notified on the
list itself instead, so each visit starts fresh, and compare by namespace
and name, so a VM of the same name in another namespace counts as a
change.

Also say which VMs lack Harvester's vmName label: the ones from Forklift
or plain manifests, not every VM created outside the UI.

Signed-off-by: Marcelo Fukumoto <marcelo.fukumoto@suse.com>

* refactor: keep the VM list's paging helpers in the component

Only canPaginate stays outside, because ResourceList calls
$loadingResources(route, store) on the component options, where there is
no `this`. The pagination headers become a computed property and the
printer column lookup a method.

Signed-off-by: Marcelo Fukumoto <marcelo.fukumoto@suse.com>

* refactor: always list VMs with PaginatedResourceTable

PaginatedResourceTable handles both modes. fetchSecondaryResources loads
every pod, volume, restore, backup, VMI and migration when the list isn't
paged, and only what every row shares when it is. VMIs without a VM are
added back through localFilter, which only runs without pagination.

The list passes the pagination context only when the cluster's API can
page, so older clusters keep the full list.

Signed-off-by: Marcelo Fukumoto <marcelo.fukumoto@suse.com>

* refactor: register the VM list's columns with the product

Both column sets go through the product's headers(), with the paged set
next to the unpaged one in config/table-headers.js. The list only adds the
node and restore progress columns, which the API can't sort or search when
paging. State sorts by the VM's Status printer column, metadata.fields.2.

Signed-off-by: Marcelo Fukumoto <marcelo.fukumoto@suse.com>

* fix: show the restart notice when coming back to the VM list

Since the list always uses PaginatedResourceTable it reads the VMs from
the store. Coming back to the list the store already holds them, so the
restart watcher never saw a change and the notice didn't show. Run it
when the list is created too.

While paging, the store can still hold the VMs another view loaded, like
every VM after Volumes or a VM's detail, so wait for the list's own page
before notifying.

Signed-off-by: Marcelo Fukumoto <marcelo.fukumoto@suse.com>

* fix: let the namespace filter know the VM list is paged

The VMs were registered for pagination with a context that only the VM
list passed. The header's namespace filter asks without one, so it didn't
know the list was paged and kept offering "Not in a Project", which the
paged API can't filter by: picking it hid every VM.

Register the VMs in every context and answer per cluster in the store's
paginationEnabled getter instead: only when the feature is on and the
API lists the VM's printer columns as metadata.fields, as the SQL cache
does. The filter, the list and the store now agree, and clusters whose
API can't page keep the full list.

The filter is shared by every list, so Not in a Project can still be
picked on a list that doesn't page. The paged VM list then asks for
another option, like the shell's generic lists do, instead of showing
no rows.

Signed-off-by: Marcelo Fukumoto <marcelo.fukumoto@suse.com>

* fix: keep the paged VM list when showing a VM's configuration

The Config tab of the Show Configuration drawer renders the VM form,
which loads every VM, PVC, PV, VMI and migration, and its device tabs
every VM again. On a paged list that replaced the list's page in the
store and emptied the list.

Hide the Config tab while the store keeps a page of VMs, so the drawer
shows the YAML. The VM's own page still shows its configuration, and a
drawer opened where every VM is loaded keeps its Config tab.

Signed-off-by: Marcelo Fukumoto <marcelo.fukumoto@suse.com>

* refactor: define the VM list's node and restore columns with the others

The node and restore progress columns are now in VM_HEADERS, so the paged
set turns their sort and search off through VM_PAGINATION_FIELDS, like the
other columns the API can't index. The list only leaves them out when they
don't apply: the node column when the cluster's nodes can't be seen, the
restore progress while no backup is being restored.

Signed-off-by: Marcelo Fukumoto <marcelo.fukumoto@suse.com>

---------

Signed-off-by: Marcelo Fukumoto <marcelo.fukumoto@suse.com>
2026-11-20 17:49:19 +01:00

440 lines
17 KiB
TypeScript

//@ts-nocheck
import {
NAMESPACE_FILTER_KINDS,
NAMESPACE_FILTER_ALL as ALL,
NAMESPACE_FILTER_ALL_ORPHANS as ALL_ORPHANS,
} from '@shell/utils/namespace-filter';
import { COUNT, MANAGEMENT, NAMESPACE } from '@shell/config/types';
import { sortBy } from '@shell/utils/sort';
import { filterBy } from '@shell/utils/array';
import { addParams } from '@shell/utils/url';
import { HCI, DATA_VOLUME } from '../../types';
import { BACKUP_TYPE } from '../../config/types';
import { HCI as HCI_ANNOTATIONS } from '../../config/labels-annotations';
import { HCI_ALLOWED_SETTINGS, HCI_SINGLE_CLUSTER_ALLOWED_SETTING } from '../../config/settings';
// ---------------------------------------------------------------------------
// Side navigation counts
//
// The side navigation (`Type.vue`, `type-map/getTree`) shows the number of
// resources of a nav item by calling `harvester/count({ name: <nav item> })`.
// Many Harvester nav items are named differently than the K8s resource they
// list (e.g. `harvesterhci.io.volume` lists `persistentvolumeclaim`, see
// `configureType(..., { resource })` in config/harvester-cluster.js). Some
// list views additionally show only a subset of that resource type.
//
// All counts come from the server: Steve's counts endpoint for nav items
// listing all resources of a type, and a filtered request with a page size of 1
// for the nav items in SERVER_FILTERED_COUNTS. Resource lists are not loaded
// for the counts: only Namespaces and Settings count their list, which is always
// loaded with the cluster.
// ---------------------------------------------------------------------------
/**
* Nav items whose list view shows only a subset of the underlying resource
* type, with the query parameters of that subset. Each query must match the
* filter of the list view, so keep them in sync with the files in list/.
*
* Steve's counts endpoint only knows the total per resource type, so these
* counts are requested from Harvester's API (`/v1/harvester`) with the query
* and a page size of 1: the `count` of the response is the number of matches
* (see `fetchFilteredCounts` in actions.ts).
*
* The queries rely on the filtering of Harvester's API, which does not use
* Steve's SQL cache: `filter=a.b=c` matches when the field contains the value,
* `filter=a.b!=` when the field is not set, and a list field matches when any
* of its items matches. Rewrite them when Harvester's API moves to the SQL
* cache: it only filters on indexed fields, and it ignores `labelSelector`
* without an error, so these counts would silently become the totals.
*/
export const SERVER_FILTERED_COUNTS: Record<string, Record<string, string>> = {
// VM backups and VM snapshots are both VirtualMachineBackup resources
[HCI.BACKUP]: { filter: `spec.type!=${ BACKUP_TYPE.SNAPSHOT }` },
[HCI.VM_SNAPSHOT]: { filter: `spec.type=${ BACKUP_TYPE.SNAPSHOT }` },
// VM backups and snapshots also create VolumeSnapshots, owned by the VirtualMachineBackup
[HCI.SNAPSHOT]: { filter: 'metadata.ownerReferences.kind=PersistentVolumeClaim' },
// Every VM image creates an internal backing image StorageClass
[HCI.STORAGE]: { filter: 'parameters.backingImage!=' },
// Only ConfigMaps labeled as cloud-init template
[HCI.CLOUD_TEMPLATE]: { labelSelector: HCI_ANNOTATIONS.CLOUD_INIT },
// Hides the storage network: Harvester creates it with this generated name, and its annotation cannot be filtered on
[HCI.NETWORK_ATTACHMENT]: { filter: 'metadata.generateName!=storagenetwork-' },
// Hides the temporary volumes CDI uses to populate a volume, which are owned by that volume
[HCI.VOLUME]: { filter: 'metadata.ownerReferences.kind!=PersistentVolumeClaim' },
// Not a nav item: the golden image volumes that the list of Volumes also hides, see `volumeCount`
[DATA_VOLUME]: { filter: 'metadata.ownerReferences.kind=VirtualMachineImage' },
};
/**
* Nav items whose list view lists the resources of all namespaces whatever the
* namespace filter is (`:ignore-filter="true"` or `:namespaced="false"` on
* `ResourceTable`). Keep in sync with the files in list/.
*/
export const IGNORE_NAMESPACE_FILTER = new Set<string>([
HCI.NETWORK_ATTACHMENT,
HCI.CLUSTER_FLOW,
HCI.CLUSTER_OUTPUT,
HCI.FLOW,
HCI.OUTPUT,
HCI.ADD_ONS,
]);
/**
* The namespaces to count for a namespaced resource type, or `null` to count
* all of them. Mirrors the namespace filter of `ResourceTable`: even with
* "All Namespaces" selected, the Harvester product (`hideSystemResources`)
* does not list resources in system namespaces, e.g. the PVCs of
* cattle-monitoring-system.
*/
export function countedNamespaces(rootGetters: any): string[] | null {
if (rootGetters.isAllNamespaces && !rootGetters.currentProduct?.hideSystemResources) {
return null;
}
// `namespaces()` computes the active namespaces on demand, so a count is available right
// away instead of only once something else has triggered that computation.
// An empty result is a valid selection (e.g. a project without namespaces): keep the
// empty array so that nothing is counted instead of the cluster-wide total.
return Object.keys(rootGetters.namespaces());
}
/**
* Whether the count of a nav item depends on the namespace filter.
*/
export function isFilteredByNamespace(name: string, schema: any): boolean {
return !!schema?.attributes?.namespaced && !IGNORE_NAMESPACE_FILTER.has(name);
}
/**
* The request for the count of a nav item in SERVER_FILTERED_COUNTS:
* - `{ url }` to request it,
* - `{ count: 0 }` if nothing can match (no namespace selected),
* - `null` if the resource type is unknown or cannot be listed.
*/
export function filteredCountRequest(getters: any, rootGetters: any, name: string): { url?: string, count?: number } | null {
const schema = getters.schemaFor(name);
if (!schema?.links?.collection) {
return null;
}
const params: Record<string, string> = { ...SERVER_FILTERED_COUNTS[name], pagesize: '1' };
if (isFilteredByNamespace(name, schema)) {
const namespaces = countedNamespaces(rootGetters);
if (namespaces?.length === 0) {
return { count: 0 };
}
if (namespaces) {
params.projectsornamespaces = namespaces.join(',');
}
}
return { url: addParams(getters.urlFor(schema.id), params) };
}
/**
* Count of all resources of `resourceType`, taken from Steve's counts
* endpoint. It is loaded with the cluster, so the count is available right
* away. `null` if the resource type is unknown.
*/
function unfilteredCount(getters: any, resourceType: string, namespaces: string[] | null): number | null {
const count = getters.all(COUNT)?.[0]?.counts?.[resourceType];
if (!count) {
return null;
}
if (!namespaces) {
return count.summary?.count || 0;
}
return namespaces.reduce((sum, namespace) => sum + (count.namespaces?.[namespace]?.count || 0), 0);
}
/**
* Count of the Namespaces list. With `hideSystemResources` it hides system and
* Fleet managed namespaces (`list/namespace.vue` of the shell), which the
* counts endpoint cannot tell apart. The namespaces are always loaded with the
* cluster (namespace filter), so the count comes from them. `null` while they
* are not loaded.
*/
function namespaceCount(getters: any): number | null {
if (!getters.haveAll(NAMESPACE)) {
return null;
}
return getters.all(NAMESPACE).filter((namespace: any) => !namespace.isSystem && !namespace.isFleetManaged).length;
}
/**
* Count of the Volumes list, which also hides golden image volumes. These are
* only marked by an annotation, which Harvester's API cannot filter on, but each
* one belongs to a DataVolume owned by its VM image. So they are counted through
* these DataVolumes and subtracted. `null` until both numbers are known.
*/
function volumeCount(state: any): number | null {
const volumes = state.filteredCounts?.[HCI.VOLUME] ?? null;
const goldenImageVolumes = state.filteredCounts?.[DATA_VOLUME] ?? null;
if (volumes === null || goldenImageVolumes === null) {
return null;
}
// A DataVolume exists briefly before its volume
return Math.max(volumes - goldenImageVolumes, 0);
}
/**
* Count of the Settings page, which shows only some of the settings (see the
* `visibleSettings` getter). `null` while the settings are not loaded.
*/
function settingCount(getters: any): number | null {
return getters.haveAll(HCI.SETTING) ? getters.visibleSettings.length : null;
}
function resourceCount(state: any, rootGetters: any, getters: any, typeObj: any): number | null {
const type = typeObj?.name;
if (!type) {
throw new Error(`Resource type required to calc count: ${ JSON.stringify(typeObj) }`);
}
if (type === HCI.VOLUME) {
return volumeCount(state);
}
// Requested by `fetchFilteredCounts` (actions.ts), `null` until the response arrives
if (SERVER_FILTERED_COUNTS[type]) {
return state.filteredCounts?.[type] ?? null;
}
if (type === NAMESPACE && rootGetters.currentProduct?.hideSystemResources) {
return namespaceCount(getters);
}
if (type === HCI.SETTING) {
return settingCount(getters);
}
// schemaFor falls back to the resource of `configureType`, so this resolves
// the nav item to the K8s resource type it lists
const schema = getters.schemaFor(type);
const resourceType = schema?.id || type;
const namespaces = isFilteredByNamespace(type, schema) ? countedNamespaces(rootGetters) : null;
return unfilteredCount(getters, resourceType, namespaces);
}
/**
* Whether this cluster's API can page the VM list. Only an API served from
* Steve's SQL cache can filter it, and that API lists the printer columns of a
* CRD as `metadata.fields[N]`. The server version alone can't tell, as it can
* be missing or not semver.
*/
function canPaginateVMs(getters: any, rootGetters: any): boolean {
if (!rootGetters['harvester-common/getFeatureEnabled']?.('vmListPagination')) {
return false;
}
const columns = getters.schemaFor(HCI.VM)?.attributes?.columns || [];
return columns.some((col: any) => col.field?.startsWith('$.metadata.fields['));
}
/**
* Wraps the store's `paginationEnabled` getter. The VM list is registered for
* pagination on every cluster (index.ts), so answer for this cluster here.
* Every caller then agrees, like the header's namespace filter, which disables
* the options a paged list can't filter by.
*/
export function vmAwarePaginationEnabled(paginationEnabled: any) {
return (state: any, getters: any, rootState: any, rootGetters: any) => (args: any) => {
const enabled = paginationEnabled(state, getters, rootState, rootGetters)(args);
const id = typeof args === 'object' ? args?.id : args;
if (!enabled || id !== HCI.VM) {
return enabled;
}
return canPaginateVMs(getters, rootGetters);
};
}
export default {
namespaceFilterOptions: (state: any, getters: any, rootState: any, rootGetters: any) => ({
addNamespace,
divider,
notFilterNamespaces
}: any) => {
const out = [{
id: ALL,
kind: NAMESPACE_FILTER_KINDS.SPECIAL,
label: rootGetters['i18n/t']('nav.ns.all'),
}];
divider(out);
const namespaces = getters.filterNamespace(notFilterNamespaces);
if (!rootGetters['isStandaloneHarvester'] && rootGetters['currentCluster'] && rootGetters['currentCluster']?.id !== '_') {
const cluster = rootGetters['currentCluster'];
let projects = rootGetters['management/all'](
MANAGEMENT.PROJECT
);
projects = sortBy(filterBy(projects, 'spec.clusterName', cluster.id), [
'nameDisplay',
]).filter((project: any) => project.nameDisplay !== 'System');
const projectsById: any = {};
const namespacesByProject: any = {};
let firstProject = true;
namespacesByProject['null'] = []; // For namespaces not in a project
for (const project of projects) {
projectsById[project.metadata.name] = project;
}
for (const namespace of namespaces) {
let projectId = namespace.projectId;
if (!projectId || !projectsById[projectId]) {
// If there's a projectId but that project doesn't exist, treat it like no project
projectId = 'null';
}
let entry = namespacesByProject[projectId];
if (!entry) {
entry = [];
namespacesByProject[namespace.projectId] = entry;
}
entry.push(namespace);
}
for (const project of projects) {
const id = project.metadata.name;
if (firstProject) {
firstProject = false;
} else {
divider(out);
}
out.push({
id: `project://${ id }`,
kind: 'project',
label: project.nameDisplay,
});
const forThisProject = namespacesByProject[id] || [];
addNamespace(out, forThisProject);
}
const orphans = namespacesByProject['null'];
if (orphans.length) {
if (!firstProject) {
divider(out);
}
out.push({
id: ALL_ORPHANS,
kind: 'project',
label: rootGetters['i18n/t']('nav.ns.orphan'),
});
addNamespace(out, orphans);
}
} else {
addNamespace(out, namespaces);
}
return out;
},
/**
* filter system/fleet/cattle namespace
*/
filterNamespace(state: any, getters: any, rootState: any, rootGetters: any, action: any) {
const allNamespaces = getters.all('namespace');
return (notFilterNamespaces: any = []) => {
return allNamespaces.filter((namespace: any) => {
return !namespace.isSystem || notFilterNamespaces.includes(namespace.id);
});
};
},
filterProject(state: any, getters: any, rootState: any, rootGetters: any) {
const projectsInAllClusters = rootGetters['management/all'](
MANAGEMENT.PROJECT
);
const currentCluster = rootGetters['currentCluster'];
const clusterId = currentCluster.id;
return projectsInAllClusters.filter((project: any) => project.spec.clusterName === clusterId && project.nameDisplay !== 'System');
},
/**
* The settings shown by the Settings page: the allowed ones that exist in the
* cluster and are supported by its version. The page and the count of its nav
* item both use this, so that they always agree.
*/
visibleSettings(state: any, getters: any, rootState: any, rootGetters: any) {
const settings = [
...(rootGetters.isSingleProduct ? rootGetters['management/all'](MANAGEMENT.SETTING) : []),
...getters.all(HCI.SETTING),
];
const settingsById = Object.fromEntries(settings.map((setting: any) => [setting.id, setting]));
const allowed = rootGetters.isStandaloneHarvester ? { ...HCI_ALLOWED_SETTINGS, ...HCI_SINGLE_CLUSTER_ALLOWED_SETTING } : HCI_ALLOWED_SETTINGS;
return Object.entries(allowed)
.filter(([id, config]: any) => settingsById[id] && (!config.featureFlag || rootGetters['harvester-common/getFeatureEnabled'](config.featureFlag)))
.map(([id, config]: any) => ({
id, config, data: settingsById[id]
}));
},
// Few harvester resources name and REAL resource are different. E.g. HCI.NETWORK_ATTACHMENT page resource is NETWORK_ATTACHMENT.
// Check in config/harvester-cluster.js for more details.
// We need to look up the schema by resource name, and fallback to find using real resource name
schemaFor: (state, getters, rootState, rootGetters) => (type, _fuzzy = false, _allowThrow = true) => {
// follow the same logic as type-map/schemaFor in /dashboard/shell/plugins/dashboard-store/getters.js
const normalizedType = getters.normalizeType(type);
const schemas = state.types['schema'];
const out = schemas?.map?.get(normalizedType);
if (out) return out;
// if not found, use the resource mapping in configureType for a second try
const resourceType = rootGetters['type-map/optionsFor'](type)?.resource;
if (resourceType && resourceType !== type) {
const normalizedResource = getters.normalizeType(resourceType);
return schemas?.map?.get(normalizedResource) || null;
}
return null;
},
// Overrides the `count` getter of the dashboard store, which looks up the
// count by the nav item name only. See "Side navigation counts" above.
count: (state: any, getters: any, rootState: any, rootGetters: any) => (typeObj: any) => {
// Like the `count` getter of the dashboard store: a nav item configured with `subTypes`
// shows the sum of the counts of these types, e.g. the Migration Overview
// (`forklift-create`) shows the number of Migration Plans.
const subTypes = rootGetters['type-map/optionsFor']?.(typeObj?.name)?.subTypes || [];
if (subTypes.length) {
return subTypes.reduce((acc: number, type: string) => acc + resourceCount(state, rootGetters, getters, { name: type }), 0);
}
return resourceCount(state, rootGetters, getters, typeObj);
},
};