BatchProcessDto

Overview #

BatchProcessDto is the data transfer object for batch processing nodes. It extends AProcessDto and adds functionality for managing multiple items/messages in a batch, handling pagination cursors, and controlling batch iteration.

Purpose:

Carry multiple items in a single batch
Manage pagination cursors for iterative processing
Add items with individual headers and limits
Control batch iteration behavior

Location: orchesty-nodejs-sdk/lib/Utils/BatchProcessDto.ts

When to Use #

You work with BatchProcessDto in every batch node:

Add items to the batch via addItem()
Manage pagination with setBatchCursor() and getBatchCursor()
Control iteration with removeBatchCursor()
Set item-specific headers and limits

Class Hierarchy #

AProcessDto<Data>
    ↓
BatchProcessDto<Data, Item>

Core Methods #

Item Management Methods #

addItem() #

public addItem(
    body: Item,
    user?: string,
    limit?: string,
    headers?: Record<string, string[] | string> | null
): BatchProcessDto<Item>

Adds a single item to the batch.

Parameters:

Parameter	Type	Description
`body`	`Item`	The item data (will be JSON-stringified if object)
`user`	`string` (optional)	User ID for this specific item
`limit`	`string` (optional)	Rate limit for this specific item
`headers`	`Record<string, string[] \| string>` (optional)	Custom headers for this item

Returns: BatchProcessDto<Item> - Batch DTO with new type

Example:

// Simple item
dto.addItem({ id: 123, name: 'John' });

// Item with user
dto.addItem({ id: 456, name: 'Jane' }, 'user-123');

// Item with rate limit
dto.addItem({ id: 789, name: 'Bob' }, undefined, 'key;60;100');

// Item with custom headers
dto.addItem({ id: 999, name: 'Alice' }, 'user-456', undefined, {
    'X-Priority': 'high'
});

setItemList() #

public setItemList(list: Item[], asBatch = false): BatchProcessDto<Item[]>

Adds multiple items at once.

Parameters:

Parameter	Type	Default	Description
`list`	`Item[]`	-	Array of items
`asBatch`	`boolean`	`false`	If true, wraps all items in single message; if false, each item is separate

Returns: BatchProcessDto<Item[]> - Batch DTO

Example:

const contacts = [
    { id: 1, name: 'John' },
    { id: 2, name: 'Jane' },
    { id: 3, name: 'Bob' }
];

// Each contact as separate message
dto.setItemList(contacts, false);

// All contacts in one message
dto.setItemList(contacts, true);

addMessage() #

public addMessage(message: IBatchMessage): this

Adds a pre-formatted message to the batch.

Parameters:

Parameter	Type	Description
`message`	`IBatchMessage`	Message with body and headers

Returns: this - Fluent interface

getMessages() #

public getMessages(): IBatchMessage[]

Returns all messages in the batch.

Returns: IBatchMessage[] - Array of batch messages

setMessages() #

public setMessages(messages: IBatchMessage[]): this

Replaces all messages.

Parameters:

Parameter	Type	Description
`messages`	`IBatchMessage[]`	New messages array

Returns: this - Fluent interface

Cursor Management Methods #

setBatchCursor() #

public setBatchCursor(cursor: string, iterateOnly = false): this

Sets a cursor for pagination/iteration. This tells the system to call the batch node again with the same cursor value.

Parameters:

Parameter	Type	Default	Description
`cursor`	`string`	-	Cursor value (page number, token, ID, etc.)
`iterateOnly`	`boolean`	`false`	If true, only iterates without calling followers until done

Returns: this - Fluent interface

Example:

// Continue pagination - followers will be called with each batch
dto.setBatchCursor('next-page-token-abc123');

// Iterate through all pages first, then call followers once at the end
dto.setBatchCursor('page-2', true);

getBatchCursor() #

public getBatchCursor(defaultValue = ''): string

Gets the current cursor value.

Parameters:

Parameter	Type	Default	Description
`defaultValue`	`string`	`''`	Value to return if cursor not set

Returns: string - Cursor value

Example:

const cursor = dto.getBatchCursor();
if (!cursor) {
    // First iteration
    console.log('Starting from beginning');
} else {
    // Continuing from cursor
    console.log(`Continuing from: ${cursor}`);
}

removeBatchCursor() #

public removeBatchCursor(): this

Removes the cursor, signaling that iteration is complete.

Returns: this - Fluent interface

Example:

if (!response.hasMore) {
    dto.removeBatchCursor(); // Done - stop iterating
}

Bridge Data Methods #

setBridgeData() #

public setBridgeData(data: string): this

Sets raw bridge data.

getBridgeData() #

public getBridgeData(): unknown

Returns the batch messages as JSON string.

Usage Examples #

Basic Item Addition #

public processAction(dto: BatchProcessDto): BatchProcessDto {
    // Add individual items
    dto.addItem({ id: 1, name: 'Item 1' });
    dto.addItem({ id: 2, name: 'Item 2' });
    dto.addItem({ id: 3, name: 'Item 3' });
    
    return dto;
}

Pagination with Cursor #

From: orchesty-nodejs-sdk/test/Batch/TestBatch.ts

const CURSOR = 'testCursor';

public processAction(dto: BatchProcessDto): BatchProcessDto {
    // Add items for this page
    dto.addItem({
        dataTest: 'testValue',
    });

    // Check if we're done iterating
    if (dto.getBatchCursor() === CURSOR) {
        dto.removeBatchCursor(); // Stop iteration
        return dto;
    }

    // Continue to next iteration
    dto.setBatchCursor(CURSOR);

    return dto;
}

API Pagination #

public async processAction(dto: BatchProcessDto): Promise<BatchProcessDto> {
    const cursor = dto.getBatchCursor();
    
    // Build URL
    let url = 'https://api.example.com/data?limit=100';
    if (cursor) {
        url += `&cursor=${cursor}`;
    }
    
    // Fetch data
    const response = await this.fetchFromAPI(url);
    
    // Add all items
    response.items.forEach(item => {
        dto.addItem(item);
    });
    
    // Handle pagination
    if (response.nextCursor) {
        dto.setBatchCursor(response.nextCursor);
    } else {
        dto.removeBatchCursor();
    }
    
    return dto;
}

Items with User Context #

public processAction(dto: BatchProcessDto): BatchProcessDto {
    const users = ['user1', 'user2', 'user3'];
    
    users.forEach(userId => {
        dto.addItem(
            { message: `Hello ${userId}` },
            userId  // Each item processed for specific user
        );
    });
    
    return dto;
}

Items with Rate Limits #

public processAction(dto: BatchProcessDto): BatchProcessDto {
    const items = this.getHighPriorityItems();
    
    items.forEach(item => {
        // Add item with custom rate limit
        dto.addItem(
            item,
            item.userId,
            `user-${item.userId};60;10`  // 10 requests per 60 seconds
        );
    });
    
    return dto;
}

Iterate-Only Mode #

public async processAction(dto: BatchProcessDto): Promise<BatchProcessDto> {
    const page = parseInt(dto.getBatchCursor() || '1', 10);
    
    const response = await this.fetchPage(page);
    
    response.items.forEach(item => {
        dto.addItem(item);
    });
    
    if (page < response.totalPages) {
        // Iterate through all pages first, call followers only at the end
        dto.setBatchCursor((page + 1).toString(), true);
    } else {
        dto.removeBatchCursor();
    }
    
    return dto;
}

Common Patterns #

Pattern 1: Simple Batch Collection #

public processAction(dto: BatchProcessDto): BatchProcessDto {
    const items = this.collectItems();
    items.forEach(item => dto.addItem(item));
    return dto;
}

Pattern 2: Cursor-Based Pagination #

public async processAction(dto: BatchProcessDto): Promise<BatchProcessDto> {
    const cursor = dto.getBatchCursor();
    const { items, nextCursor } = await this.fetchPage(cursor);
    
    items.forEach(item => dto.addItem(item));
    
    if (nextCursor) {
        dto.setBatchCursor(nextCursor);
    } else {
        dto.removeBatchCursor();
    }
    
    return dto;
}

Pattern 3: Conditional Iteration #

public async processAction(dto: BatchProcessDto): Promise<BatchProcessDto> {
    const response = await this.fetchData();
    
    response.data.forEach(item => dto.addItem(item));
    
    // Only continue if conditions are met
    if (response.hasMore && response.data.length > 0) {
        dto.setBatchCursor(response.nextToken);
    } else {
        dto.removeBatchCursor();
    }
    
    return dto;
}

Next Steps #

ABatchNode - Create batch processing nodes
ProcessDto - Standard data handling
AConnector - Create regular connectors

BatchProcessDto

Overview #

When to Use #

Class Hierarchy #

Core Methods #

Item Management Methods #

addItem() #

setItemList() #

addMessage() #

getMessages() #

setMessages() #

Cursor Management Methods #

setBatchCursor() #

getBatchCursor() #

removeBatchCursor() #

Bridge Data Methods #

setBridgeData() #

getBridgeData() #

Usage Examples #

Basic Item Addition #

Pagination with Cursor #

API Pagination #

Items with User Context #

Items with Rate Limits #

Iterate-Only Mode #

Common Patterns #

Pattern 1: Simple Batch Collection #

Pattern 2: Cursor-Based Pagination #

Pattern 3: Conditional Iteration #

Next Steps #

See Also #