Efficient Large Dataset Management

The 7G Registry Platform manages massive datasets across SMSF entities, investor transactions, and compliance records. Smart pagination ensures optimal performance while handling thousands of financial entities and transactions.

Performance Control

Prevent timeouts when processing large SMSF portfolios or bulk transaction reporting for Australian compliance.

📊

Scalable Processing

Balance server resources while handling enterprise-grade Australian financial services data volumes.

🎯

Optimal User Experience

Enable responsive financial dashboards and reporting interfaces with progressive data loading.

7G Pagination System

📋

System Limits

Maximum: 1,000 records per page | Default: Varies by endpoint (Distribution/Account defaults to 1,000) | Indexing: 1-based page numbers

💡

Detection Strategy

Last page reached when recordCount < pageSize. Always include both pageSize and pageNumber for clarity and consistency across all paginated requests.

Pagination Parameters

Parameter Type Required Description
pageSize
integer
Optional
Records per page. Defaults vary by endpoint (Distribution/Account defaults to 1,000). Maximum: 1,000 records.
pageNumber
integer
Optional
Page number to retrieve using 1-based indexing (starts from 1, not 0)
pageSize
integer
Optional
Records per page. Defaults vary by endpoint (Distribution/Account defaults to 1,000). Maximum: 1,000 records.
pageNumber
integer
Optional
Page number to retrieve using 1-based indexing (starts from 1, not 0)

Response Structure

Standard paginated response format for all 7G endpoints

💡

Key Insight

recordCount shows current page size, not total records. When recordCount < pageSize, you've reached the last page.

json
// GET /BizEntity?pageSize=25&pageNumber=2
{
  "result": true,
  "message": "",
  "recordCount": 25,  // Current page size
  "data": [
    {
      "bizEntityID": 12345,
      "name": "Smith Super Fund",
      "bizEntityTypeID": 4
    }
    // ... 24 more SMSF entities
  ]
}

Common Pagination Patterns

Practical approaches for Australian financial services workloads

UI Display

25-50 records

Financial dashboards, SMSF entity lists, and transaction tables for user interfaces.

json
// SMSF dashboard with moderate page size
GET /BizEntity?bizEntityTypeID=4&pageSize=25&pageNumber=1

// Entity dropdown population  
GET /BizEntity?bizEntityStatusID=1&pageSize=50&pageNumber=1

Bulk Processing

500-1000 records

Regulatory reporting, data synchronization, and compliance batch processing.

json
// Compliance reporting with large page sizes
GET /BizTransaction?transactionDate.GreaterThan=2024-01-01&pageSize=1000&pageNumber=1

// Process all SMSF entities for regulatory export
async function exportAllSMSFs() {
  let page = 1;
  let hasMore = true;

  while (hasMore) {
    const response = await fetch(
      `/BizEntity?bizEntityTypeID=4&pageSize=1000&pageNumber=${page}`
    );
    const data = await response.json();

    await processForExport(data.data);
    hasMore = data.recordCount === 1000;
    page++;
  }
}

Incremental Sync

100-250 records

Real-time monitoring, change detection, and incremental updates for compliance tracking.

json
// Monitor recent SMSF entity changes
GET /BizEntity?bizEntityTypeID=4&updatedDate.GreaterThan=2024-12-15T10:00:00Z&pageSize=100&pageNumber=1

// Incremental transaction sync
async function syncRecentTransactions(lastSyncTime) {
  const changes = [];
  let page = 1;
  let hasMore = true;
  
  while (hasMore) {
    const response = await fetch(
      `/BizTransaction?updatedDateFull.GreaterThan=${lastSyncTime}&pageSize=250&pageNumber=${page}`
    );
    const data = await response.json();
    
    changes.push(...data.data);
    hasMore = data.recordCount === 250;
    page++;
  }
  
  return changes;
}

Performance & Best Practices

Optimize pagination for Australian financial services workloads

Right-Size Page Limits

25-50 records → UI displays, entity dropdowns
100-250 records → Reports, monitoring dashboards
500-1000 records → Bulk processing, compliance exports

Smart Filtering Examples

Date-based filtering: ?updatedDate.GreaterThan=2024-01-01&pageSize=500&pageNumber=1
Entity type filtering: ?bizEntityTypeID.In=1,4&pageSize=250&pageNumber=1
Status filtering: ?bizEntityStatusID=1&pageSize=100&pageNumber=1

Essential Guidelines

  • Always include both parameters: Use pageSize AND pageNumber for consistency
  • Check recordCount: When recordCount < pageSize, you've reached the end
  • 1-based indexing: Page numbers start from 1, not 0
  • Combine with filters: Reduce dataset size before paginating
  • Handle timeouts: Larger page sizes may timeout on slow networks
  • Cache strategically: Store recent pages for better UX

Quick Reference

Paginated Endpoints

/BizEntity /BizTransaction /BizEvent /Investment /Distribution/Account

All GET endpoints supporting paginated requests with pageSize/pageNumber parameters

System Limits

ParameterDefaultMaximumNotes
pageSizeVaries1,000Defaults vary by endpoint
pageNumber1No limit1-based indexing