Operations · 3 min read

BulkDelete

This method deletes rows from the database in bulk, matched by the defined qualifiers. It is supported for Aurora PostgreSQL. Aurora PostgreSQL also has a dedicated BulkDeleteByKey operation for deleting by primary key.

Call Flow Diagram

The diagram below shows the flow when calling this operation.

flowchart TD
    Client["Client<br/>(RepoDB)"] -->|BulkDelete| Source["Entities /<br/>DataTable /<br/>DbDataReader"]
    Source --> Pseudo["Create Pseudo Table<br/>(Memory by default,<br/>qualifier columns only) + index"]
    Pseudo --> Copy["AuroraDbBulkCopy"]
    Copy -->|Write| PseudoTable[("Pseudo Table")]
    PseudoTable -->|"DELETE ... USING Pseudo<br/>WHERE (qualifiers match)"| Table[("Target Table")]
    PseudoTable -->|Drop| Cleanup(["Pseudo Table Dropped"])

Use Case

Use this method to delete rows at high speed. It leverages AuroraDbBulkCopy, a native binary COPY-based bulk writer from RepoDb.Connector.AuroraDb.Npgsql.

For deleting 1,000 or more rows, prefer this method over DeleteAll.

A pseudo (staging) table — holding only the qualifier columns, with an index on them — is always created first. The library writes to it via AuroraDbBulkCopy internally, then cascades the deletions to the target table via a DELETE ... USING statement — see Operations (AuroraDB PostgreSQL) for the underlying mechanics.

Special Arguments

The qualifiers, bulkCopyTimeout, batchSize and pseudoTableType arguments are available for this operation.

qualifiers defines the fields used to match existing rows, corresponding to the WHERE clause. Defaults to the primary column if not specified.

bulkCopyTimeout overrides the command timeout, in seconds.

batchSize overrides the number of rows sent to the server per batch. When not set, all items are sent at once.

pseudoTableType (via AuroraDbBulkImportPseudoTableType) controls the kind of staging table used internally.

Caveats

This operation creates a pseudo-temporary table internally. The database user must have permission to create tables (or the TEMPORARY privilege on the database, for Memory staging), or the underlying exception from Aurora PostgreSQL will surface from the create-table statement.

Usability

The following example retrieves all inactive people, then bulk-deletes them from the Person table.

using (var connection = new AuroraDbConnection(connectionString))
{
    var people = connection.Query<Person>(e => e.IsActive == false);
    var deletedRows = connection.BulkDelete<Person>(people);
}

To specify a batch size:

using (var connection = new AuroraDbConnection(connectionString))
{
    var deletedRows = connection.BulkDelete<Person>(people, batchSize: 100);
}

When batchSize is not set, all rows are sent to the server in a single batch.

DataTable

using (var connection = new AuroraDbConnection(connectionString))
{
    var table = ConvertToDataTable(people);
    var deletedRows = connection.BulkDelete("Person", table);
}

Dictionary/ExpandoObject

using (var sourceConnection = new AuroraDbConnection(sourceConnectionString))
{
    var result = sourceConnection.QueryAll("Person");
    using (var destinationConnection = new AuroraDbConnection(destinationConnectionString))
    {
        var deletedRows = destinationConnection.BulkDelete("Person", result);
    }
}

DataReader

using (var sourceConnection = new AuroraDbConnection(sourceConnectionString))
{
    using (var reader = sourceConnection.ExecuteReader("SELECT * FROM \"Person\";"))
    {
        using (var destinationConnection = new AuroraDbConnection(destinationConnectionString))
        {
            var rows = destinationConnection.BulkDelete("Person", reader);
        }
    }
}

Targeting a Table

To target a specific table, pass the literal table name.

using (var connection = new AuroraDbConnection(connectionString))
{
    var deletedRows = connection.BulkDelete("Person", people);
}

Field Qualifiers

By default, the primary column is used as the qualifier. To override, pass a list of Field objects in the qualifiers argument.

using (var connection = new AuroraDbConnection(connectionString))
{
    var deletedRows = connection.BulkDelete<Person>(people,
        qualifiers: e => new { e.LastName, e.DateOfBirth });
}

Use indexed columns from the target table as qualifiers to maximize performance.

Async Method

An equivalent BulkDeleteAsync method is also available.

using (var connection = new AuroraDbConnection(connectionString))
{
    var people = connection.Query<Person>(e => e.IsActive == false);
    var deletedRows = await connection.BulkDeleteAsync<Person>(people);
}