relstorage.adapters.postgresql.mover – IObjectMover implementation

IObjectMover implementation.

class PostgreSQLObjectMover(*args, **kwargs)

Bases: AbstractObjectMover

Parameters:

database_driver – The IDBDriver in use.

download_blob(cursor, oid, tid, filename)

Download a blob into a file.

on_store_opened(cursor, restart=False)

Create the temporary tables for storing objects

replace_temps(cursor, state_oid_tid_iter)

Assumes that store_temps is using an upsert query and simply calls that method.

The same comments apply. In particular, MySQLclient won’t optimize an UPDATE in the same way it does an INSERT.

restore(cursor, batcher, oid, tid, data)

Store an object directly, without conflict detection.

Used for copying transactions into this database.

store_temps(cursor, state_oid_tid_iter)

Uses the cursor’s executemany method to store temporary objects.

Parameters:

state_oid_tid_iter – An iterable over tuples (state, oid_int, tid_int). Data may be None to indicate we should store a NULL.

If there is a more optimal way to implement putting objects in the database, please do so.

• On SQLite, executemany is implemnted in a C looping over the provided iterator. Which it turns out is exactly what the normal execute method also does (it just uses a one-row iterator). So executemany that saves substantial setup overhead dealing with sqlite’s prepared statements.

• On Postgresql, we use COPY for this (unless we’re using the ‘gevent psycopg2’ driver; it’s the only thing that doesn’t support COPY). None of the supported PostgreSQL drivers have a good executemany method, so they should fall back to using our own RowBatcher.

• On Oracle, we use the RowBatcher with a combination of bulk array operations and direct inserts.

• On MySQL, the preferred driver (mysqlclient) has a decent implementation of executemany for INSERT or REPLACE (basically an optimized form of what our RowBatcher does). That implementation is shared with PyMySQL as well, but it must be a simple INSERT statement matching a regular expression. Note that it has a bug though: it can’t handle an iterator that’s empty.

upload_blob(cursor, oid, tid, filename)

Upload a blob from a file.

If serial is None, upload to the temporary table.

class PostgreSQLRowBatcherStoreTemps(keep_history, binary, batcher_factory=<class 'relstorage.adapters.batch.RowBatcher'>)

Bases: RowBatcherStoreTemps

class TempStoreCopyBuffer(table, state_oid_tid_iterable, digester)

Bases: BufferedIOBase

A binary file-like object for putting data into temp_store.

read(size=-1)

Read and return up to n bytes.

If the size argument is omitted, None, or negative, read and return all data until EOF.

If the size argument is positive, and the underlying raw stream is not ‘interactive’, multiple raw reads may be issued to satisfy the byte count (unless EOF is reached first). However, for interactive raw streams (as well as sockets and pipes), at most one raw read will be issued, and a short result does not imply that EOF is imminent.

Return an empty bytes object on EOF.

Return None if the underlying raw stream was open in non-blocking mode and no data is available at the moment.