Skip to main content

@idpass/data-collect-core / EntityDataManager

Class: EntityDataManager

Defined in: components/EntityDataManager.ts:141

Primary API interface for the ID PASS DataCollect library.

The EntityDataManager orchestrates all data operations including:

  • Form submission and event processing
  • Entity creation, modification, and querying
  • Data synchronization with remote servers and external systems
  • Audit trail management and duplicate detection

This class implements the Command Query Responsibility Segregation (CQRS) pattern, where all changes go through events (FormSubmissions) and queries access the current state through the EntityStore.

Examples​

Basic usage:

import { EntityDataManager, EntityType, SyncLevel } from '@idpass/data-collect-core';

// Initialize the manager (typically done once)
const manager = new EntityDataManager(
  eventStore,
  entityStore,
  eventApplierService,
  externalSyncManager,
  internalSyncManager
);

// Create a new individual
const individual = await manager.submitForm({
  guid: uuidv4(),
  entityGuid: uuidv4(),
  type: 'create-individual',
  data: { name: 'John Doe', age: 30 },
  timestamp: new Date().toISOString(),
  userId: 'user-123',
  syncLevel: SyncLevel.LOCAL
});

// Create a household group
const group = await manager.submitForm({
  guid: uuidv4(),
  entityGuid: uuidv4(),
  type: 'create-group',
  data: { name: 'Smith Family' },
  timestamp: new Date().toISOString(),
  userId: 'user-123',
  syncLevel: SyncLevel.LOCAL
});

// Add individual to group
await manager.submitForm({
  guid: uuidv4(),
  entityGuid: group.guid,
  type: 'add-member',
  data: { members: [{ guid: individual.guid, name: individual.data.name }] },
  timestamp: new Date().toISOString(),
  userId: 'user-123',
  syncLevel: SyncLevel.LOCAL
});

Offline-first usage:

// Works completely offline
const offlineManager = new EntityDataManager(
  eventStore,
  entityStore,
  eventApplierService
  // No sync managers - offline only
);

// All operations work locally
const entity = await offlineManager.submitForm(formData);
const allEntities = await offlineManager.getAllEntities();

With synchronization:

// Sync with remote server
if (manager.hasInternalSyncManager()) {
  await manager.sync();
}

// Check sync status
if (manager.isSyncing()) {
  console.log('Sync in progress...');
}

Constructors​

Constructor​

new EntityDataManager(eventStore, entityStore, eventApplierService, externalSyncManager?, internalSyncManager?, authManager?): EntityDataManager

Defined in: components/EntityDataManager.ts:154

Creates a new EntityDataManager instance.

Parameters​

eventStore​

EventStore

Store for managing events/form submissions.

entityStore​

EntityStore

Store for managing current entity state.

eventApplierService​

EventApplierService

Service for applying events to entities.

externalSyncManager?​

ExternalSyncManager

Optional manager for external system sync.

internalSyncManager?​

InternalSyncManager

Optional manager for server sync.

authManager?​

AuthManager

Optional manager for authentication.

Returns​

EntityDataManager

Accessors​

lastRefreshToken​

Get Signature​

get lastRefreshToken(): string | null

Defined in: components/EntityDataManager.ts:167

Returns the last refresh token received from the sync server after login. Used by mobile clients to store the token for offline silent re-authentication.

Returns​

string | null

Methods​

isSyncing()​

isSyncing(): boolean

Defined in: components/EntityDataManager.ts:176

Checks if a synchronization operation is currently in progress.

Returns​

boolean

True if sync is active, false otherwise.

isExternalSyncing()​

isExternalSyncing(): boolean

Defined in: components/EntityDataManager.ts:188

Checks if an external synchronization operation is currently in progress.

Returns​

boolean

True if external sync is active, false otherwise.

submitForm()​

submitForm(formData, options?): Promise<EntityDoc | null>

Defined in: components/EntityDataManager.ts:228

Submits a form to create or modify entities through the event sourcing system.

All modifications go through events (FormSubmissions) which are applied to create the new state.

Parameters​

formData​

FormSubmission

The form submission containing the event data.

options?​

SubmitFormOptions

Returns​

Promise<EntityDoc | null>

The resulting entity after applying the event, or null if creation failed.

Example​

// Create a new individual
const individual = await manager.submitForm({
  guid: uuidv4(),
  entityGuid: uuidv4(),
  type: 'create-individual',
  data: { name: 'John Doe', age: 30, email: '[email protected]' },
  timestamp: new Date().toISOString(),
  userId: 'user-123',
  syncLevel: SyncLevel.LOCAL
});

// Update an existing individual
const updated = await manager.submitForm({
  guid: uuidv4(),
  entityGuid: individual.guid,
  type: 'update-individual',
  data: { age: 31 }, // Only changed fields
  timestamp: new Date().toISOString(),
  userId: 'user-123',
  syncLevel: SyncLevel.LOCAL
});

submitFormBatch()​

submitFormBatch(events, options?): Promise<{ success: boolean; applied: number; failed: FormSubmission[]; errors: string[]; }>

Defined in: components/EntityDataManager.ts:258

Submits a batch of form submissions atomically.

All events in the batch are processed in sequence. If any event fails, the error is thrown immediately and the caller is responsible for handling the partial state. The response includes the count of successfully applied events and details about the failing event.

This method is intended for server-side sync push handlers where it is critical that the client receives an accurate success or failure signal. On failure the client should NOT advance its sync cursor so that the failed events can be retried on the next push.

Parameters​

events​

FormSubmission[]

The ordered list of form submissions to process.

options?​

SubmitFormOptions

Optional apply-path options forwarded to every event (e.g. the member sub-write scope guard for bounded /api/sync/push callers).

Returns​

Promise<{ success: boolean; applied: number; failed: FormSubmission[]; errors: string[]; }>

An object describing the outcome: applied count and any error details.

Example​

const result = await manager.submitFormBatch(events);
if (!result.success) {
  console.error(`Batch failed after ${result.applied} events: ${result.errors}`);
}

getAllEvents()​

getAllEvents(options?): Promise<FormSubmission[]>

Defined in: components/EntityDataManager.ts:300

Retrieves all events (form submissions) from the event store.

Provides access to the complete audit trail of all changes made to entities. Events are returned in chronological order.

Parameters​

options?​

ReadAuditOptions = {}

Returns​

Promise<FormSubmission[]>

Array of all form submissions/events.

Example​

const events = await manager.getAllEvents();

// Filter events by type
const createEvents = events.filter(e => e.type.startsWith('create-'));

// Filter events by user
const userEvents = events.filter(e => e.userId === 'user-123');

// Filter events by entity
const entityEvents = events.filter(e => e.entityGuid === 'entity-456');

getAllEntities()​

getAllEntities(options?): Promise<EntityPair[]>

Defined in: components/EntityDataManager.ts:334

Retrieves all entities from the entity store.

Returns EntityPairs containing both the initial state (when first loaded/synced) and the current modified state. This enables change tracking and conflict resolution.

Parameters​

options?​

ReadAuditOptions = {}

Returns​

Promise<EntityPair[]>

Array of entity pairs with initial and current state.

Example​

const entities = await manager.getAllEntities();

// Find entities that have been modified locally
const modifiedEntities = entities.filter(pair =>
  pair.initial.version !== pair.modified.version
);

// Get only individuals
const individuals = entities.filter(pair =>
  pair.modified.type === EntityType.Individual
);

// Get only groups
const groups = entities.filter(pair =>
  pair.modified.type === EntityType.Group
);

getEntity()​

getEntity(id, options?): Promise<EntityPair>

Defined in: components/EntityDataManager.ts:371

Retrieves a specific entity by its internal database ID.

Parameters​

id​

string

Internal database ID of the entity.

options?​

ReadAuditOptions = {}

Returns​

Promise<EntityPair>

Entity pair with initial and current state.

Throws​

When entity is not found.

Example​

try {
  const entityPair = await manager.getEntity('entity-123');

  console.log('Original state:', entityPair.initial);
  console.log('Current state:', entityPair.modified);

  // Check if entity has been modified
  const hasChanges = entityPair.initial.version !== entityPair.modified.version;

  if (entityPair.modified.type === EntityType.Group) {
    const group = entityPair.modified as GroupDoc;
    console.log('Group members:', group.memberIds);
  }
} catch (error) {
  if (error instanceof AppError && error.code === 'ENTITY_NOT_FOUND') {
    console.log('Entity does not exist');
  }
}

getMembers()​

getMembers(groupId): Promise<EntityPair[]>

Defined in: components/EntityDataManager.ts:491

Retrieves all members of a specific group.

Parameters​

groupId​

string

Internal database ID of the group.

Returns​

Promise<EntityPair[]>

Array of entity pairs for all group members.

Throws​

When group is not found or entity is not a group.

Example​

try {
  const members = await manager.getMembers('group-123');
  console.log(`Group has ${members.length} members`);

  members.forEach(member => {
    console.log(`Member: ${member.modified.data.name}`);
  });
} catch (error) {
  if (error instanceof AppError && error.code === 'INVALID_GROUP') {
    console.log('Group not found or invalid');
  }
}

hasUnsyncedEvents()​

hasUnsyncedEvents(): Promise<boolean>

Defined in: components/EntityDataManager.ts:516

Checks if there are any unsynced events waiting to be synchronized.

Only available when an InternalSyncManager is configured.

Returns​

Promise<boolean>

True if there are unsynced events, false otherwise.

Example​

if (await manager.hasUnsyncedEvents()) {
  console.log('There are changes waiting to sync');
  await manager.syncWithSyncServer();
}

getUnsyncedEventsCount()​

getUnsyncedEventsCount(): Promise<number>

Defined in: components/EntityDataManager.ts:540

Gets the count of unsynced events waiting to be synchronized.

Only available when an InternalSyncManager is configured.

Returns​

Promise<number>

Number of unsynced events.

Example​

const count = await manager.getUnsyncedEventsCount();
console.log(`${count} events waiting to sync`);

if (count > 100) {
  console.log('Large number of changes - consider syncing soon');
}

syncWithSyncServer()​

syncWithSyncServer(): Promise<void>

Defined in: components/EntityDataManager.ts:568

Synchronizes local data with the remote sync server.

Performs a full bidirectional sync: pushes local changes to server and pulls remote changes to local storage.

Only available when an InternalSyncManager is configured.

Returns​

Promise<void>

Throws​

When sync fails or authentication is required.

Example​

try {
  console.log('Starting sync...');
  await manager.syncWithSyncServer();
  console.log('Sync completed successfully');
} catch (error) {
  console.error('Sync failed:', error.message);
}

searchEntities()​

searchEntities(criteria, options?): Promise<EntityPair[]>

Defined in: components/EntityDataManager.ts:597

Searches entities using specified criteria.

Parameters​

criteria​

SearchCriteria

Search criteria array with query conditions.

options?​

ReadAuditOptions = {}

Returns​

Promise<EntityPair[]>

Array of entity pairs matching the criteria.

Example​

// Search for adults
const adults = await manager.searchEntities([
  { "data.age": { $gte: 18 } },
  { "type": "individual" }
]);

// Search for groups with specific name
const smithFamilies = await manager.searchEntities([
  { "data.name": { $regex: /smith/i } },
  { "type": "group" }
]);

// TODO: Document the exact query syntax supported

getAuditTrailByEntityGuid()​

getAuditTrailByEntityGuid(entityGuid, options?): Promise<AuditLogEntry[]>

Defined in: components/EntityDataManager.ts:629

Retrieves the complete audit trail for a specific entity.

Parameters​

entityGuid​

string

Global unique identifier of the entity.

options?​

ReadAuditOptions = {}

Returns​

Promise<AuditLogEntry[]>

Array of audit log entries in chronological order.

Throws​

When audit trail retrieval fails.

Example​

try {
  const auditTrail = await manager.getAuditTrailByEntityGuid('entity-123');

  console.log(`Found ${auditTrail.length} audit entries`);
  auditTrail.forEach(entry => {
    console.log(`${entry.timestamp}: ${entry.action} by ${entry.userId}`);
  });
} catch (error) {
  if (error instanceof AppError && error.code === 'AUDIT_TRAIL_ERROR') {
    console.error('Failed to retrieve audit trail');
  }
}

getEventsSince()​

getEventsSince(timestamp): Promise<FormSubmission[]>

Defined in: components/EntityDataManager.ts:686

Retrieves events created since a specific timestamp.

Useful for incremental sync operations and change tracking.

Parameters​

timestamp​

string

ISO timestamp to filter events from.

Returns​

Promise<FormSubmission[]>

Array of events created after the specified timestamp.

Example​

const lastSync = '2024-01-01T00:00:00Z';
const recentEvents = await manager.getEventsSince(lastSync);

console.log(`${recentEvents.length} events since last sync`);
recentEvents.forEach(event => {
  console.log(`${event.timestamp}: ${event.type} on ${event.entityGuid}`);
});

getEventsSincePagination()​

getEventsSincePagination(timestamp, limit): Promise<{ events: FormSubmission[]; nextCursor: string | Date | null; }>

Defined in: components/EntityDataManager.ts:711

Retrieves events since a timestamp with pagination support.

Parameters​

timestamp​

string

ISO timestamp to filter events from.

limit​

number

Maximum number of events to return (default: 10).

Returns​

Promise<{ events: FormSubmission[]; nextCursor: string | Date | null; }>

Object with events array and cursor for next page.

Example​

let cursor = '2024-01-01T00:00:00Z';
let allEvents: FormSubmission[] = [];

do {
  const result = await manager.getEventsSincePagination(cursor, 50);
  allEvents.push(...result.events);
  cursor = result.nextCursor;
} while (cursor);

console.log(`Retrieved ${allEvents.length} events total`);

closeConnection()​

closeConnection(): Promise<void>

Defined in: components/EntityDataManager.ts:731

Closes all database connections and cleans up resources.

Should be called when the EntityDataManager is no longer needed to properly release database connections and prevent memory leaks.

Returns​

Promise<void>

Example​

// At application shutdown
await manager.closeConnection();
console.log('Database connections closed');

clearStore()​

clearStore(): Promise<void>

Defined in: components/EntityDataManager.ts:751

Clears all data from both entity and event stores.

⚠️ WARNING: This permanently deletes all data! Only use for testing or when intentionally resetting the system.

Returns​

Promise<void>

Example​

// For testing only
if (process.env.NODE_ENV === 'test') {
  await manager.clearStore();
  console.log('Test data cleared');
}

purgeEntitiesNotIn()​

purgeEntitiesNotIn(keepGuids): Promise<{ purgedEntities: number; purgedEvents: number; }>

Defined in: components/EntityDataManager.ts:777

Drops local entities outside keepGuids and any events targeting them.

Used by the client when the server advertises a new scope hash on /pull — out-of-scope entities are purged before the next pull's events are applied. Returns counts for telemetry / UI surfacing.

Purge does NOT generate delete-entity events. It is a local data- minimization operation, not a state change visible to the server.

Parameters​

keepGuids​

readonly string[]

The entity guids that should remain after the purge.

Returns​

Promise<{ purgedEntities: number; purgedEvents: number; }>

Counts of purged entities and events.

Example​

// After receiving a new scopeHash on /pull, retain only allowed entities.
const allowed = pullResponse.scope?.allowedEntityGuids ?? [];
const result = await manager.purgeEntitiesNotIn(allowed);
console.log(`Removed ${result.purgedEntities} entities and ${result.purgedEvents} events.`);

saveAuditLogs()​

saveAuditLogs(auditLogs): Promise<void>

Defined in: components/EntityDataManager.ts:818

Saves multiple audit log entries to the event store.

Parameters​

auditLogs​

AuditLogEntry[]

Array of audit log entries to save.

Returns​

Promise<void>

Example​

const auditLogs: AuditLogEntry[] = [
  {
    guid: 'audit-1',
    timestamp: '2024-01-01T12:00:00Z',
    userId: 'user-123',
    action: 'create-individual',
    eventGuid: 'event-456',
    entityGuid: 'person-789',
    changes: { name: 'John Doe' },
    signature: 'sha256:abc123'
  }
];

await manager.saveAuditLogs(auditLogs);

getAuditLogsSince()​

getAuditLogsSince(since): Promise<AuditLogEntry[]>

Defined in: components/EntityDataManager.ts:836

Retrieves audit logs created since a specific timestamp.

Parameters​

since​

string

ISO timestamp to filter audit logs from.

Returns​

Promise<AuditLogEntry[]>

Array of audit log entries created after the specified timestamp.

Example​

const lastAuditSync = '2024-01-01T00:00:00Z';
const recentAudits = await manager.getAuditLogsSince(lastAuditSync);

console.log(`${recentAudits.length} audit entries since last sync`);

getPotentialDuplicates()​

getPotentialDuplicates(): Promise<object[]>

Defined in: components/EntityDataManager.ts:865

Retrieves all potential duplicate entity pairs detected by the system.

Returns​

Promise<object[]>

Array of entity GUID pairs that are potential duplicates.

Example​

const duplicates = await manager.getPotentialDuplicates();

if (duplicates.length > 0) {
  console.log(`Found ${duplicates.length} potential duplicate pairs`);

  for (const pair of duplicates) {
    const entity1 = await manager.getEntity(pair.entityGuid);
    const entity2 = await manager.getEntity(pair.duplicateGuid);

    console.log('Potential duplicate:');
    console.log('Entity 1:', entity1.modified.data);
    console.log('Entity 2:', entity2.modified.data);

    // TODO: Implement duplicate resolution workflow
  }
}

syncWithExternalSystem()​

syncWithExternalSystem(credentials?, options?): Promise<SyncResult | undefined>

Defined in: components/EntityDataManager.ts:892

Synchronizes data with external systems (e.g., OpenSPP, custom APIs).

Parameters​

credentials?​

ExternalSyncCredentials

Optional credentials for external system authentication.

options?​

SyncOptions

Returns​

Promise<SyncResult | undefined>

Throws​

When external sync fails or credentials are invalid.

Example​

// Sync with OpenSPP system
try {
  await manager.syncWithExternalSystem({
    username: 'sync_user',
    password: 'sync_password'
  });
  console.log('External sync completed');
} catch (error) {
  console.error('External sync failed:', error.message);
}

// Sync without credentials (if configured in adapter)
await manager.syncWithExternalSystem();

login()​

login(credentials, type?): Promise<void>

Defined in: components/EntityDataManager.ts:947

Logs in the user via the configured AuthManager.

Parameters​

credentials​

PasswordCredentials | TokenCredentials | null

The credentials for login.

type?​

string

Optional. The type of authentication provider to use.

Returns​

Promise<void>

A Promise that resolves when the login operation is complete.

Throws​

If AuthManager is not configured or login fails.

initializeAuthManager()​

initializeAuthManager(): Promise<void>

Defined in: components/EntityDataManager.ts:959

Initializes the AuthManager.

Returns​

Promise<void>

A Promise that resolves when the AuthManager is initialized.

Throws​

If AuthManager is not configured.

logout()​

logout(): Promise<void>

Defined in: components/EntityDataManager.ts:971

Logs out the user via the configured AuthManager.

Returns​

Promise<void>

A Promise that resolves when the logout operation is complete.

Throws​

If AuthManager is not configured or logout fails.

validateToken()​

validateToken(type, token): Promise<boolean>

Defined in: components/EntityDataManager.ts:985

Validates an authentication token via the configured AuthManager.

Parameters​

type​

string

The type of authentication provider.

token​

string

The token string to validate.

Returns​

Promise<boolean>

A Promise that resolves to true if the token is valid, false otherwise.

Throws​

If AuthManager is not configured.

isAuthenticated()​

isAuthenticated(): Promise<boolean>

Defined in: components/EntityDataManager.ts:998

Checks if the user is authenticated via the configured AuthManager.

Returns​

Promise<boolean>

A Promise that resolves to true if authenticated, false otherwise.

Throws​

If AuthManager is not configured.

handleCallback()​

handleCallback(type): Promise<void>

Defined in: components/EntityDataManager.ts:1012

Handles the authentication callback via the configured AuthManager.

Parameters​

type​

string

The type of authentication provider.

Returns​

Promise<void>

A Promise that resolves when the callback is handled.

Throws​

If AuthManager is not configured.