relstorage.adapters.mover – IObjectMover implementation

IObjectMover implementation.

class AbstractObjectMover(database_driver, options, runner=None, version_detector=None, batcher_factory=<class 'relstorage.adapters.batch.RowBatcher'>)

Bases: DatabaseHelpersMixin, ABC

Parameters:

database_driver – The IDBDriver in use.

current_object_tids(cursor, oids, timeout=None)

Returns the current {oid: tid} for specified object ids.

download_blob(cursor, oid, tid, filename)

Download a blob into a file.

exists(cursor, oid)

Returns a true value if the given object exists.

get_object_tid_after(cursor, oid, tid)

Returns the tid of the next change after an object revision.

Returns None if no later state exists.

load_before(cursor, oid, tid)

Returns the pickle and tid of an object before transaction tid.

Returns (None, None) if no earlier state exists.

load_current(cursor, oid)

Returns the current pickle and integer tid for an object.

oid is an integer. Returns (None, None) if object does not exist.

load_currents(cursor, oids)

Returns the current (oid, state, tid) for specified object ids.

load_revision(cursor, oid, tid)

Returns the pickle for an object on a particular transaction.

Returns None if no such state exists.

move_from_temp(cursor, tid, txn_has_blobs)

Move the temporarily stored objects to permanent storage.

on_load_opened(cursor, restart=False)

Hook for subclasses.

on_store_opened(cursor, restart=False)

Hook for subclasses.

replace_temps(cursor, state_oid_tid_iter)

Assumes that store_temps is using an upsert query and simply calls that method.

The same comments apply. In particular, MySQLclient won’t optimize an UPDATE in the same way it does an INSERT.

store_temps(cursor, state_oid_tid_iter)

Uses the cursor’s executemany method to store temporary objects.

Parameters:

state_oid_tid_iter – An iterable over tuples (state, oid_int, tid_int). Data may be None to indicate we should store a NULL.

If there is a more optimal way to implement putting objects in the database, please do so.

• On SQLite, executemany is implemnted in a C looping over the provided iterator. Which it turns out is exactly what the normal execute method also does (it just uses a one-row iterator). So executemany that saves substantial setup overhead dealing with sqlite’s prepared statements.

• On Postgresql, we use COPY for this (unless we’re using the ‘gevent psycopg2’ driver; it’s the only thing that doesn’t support COPY). None of the supported PostgreSQL drivers have a good executemany method, so they should fall back to using our own RowBatcher.

• On Oracle, we use the RowBatcher with a combination of bulk array operations and direct inserts.

• On MySQL, the preferred driver (mysqlclient) has a decent implementation of executemany for INSERT or REPLACE (basically an optimized form of what our RowBatcher does). That implementation is shared with PyMySQL as well, but it must be a simple INSERT statement matching a regular expression. Note that it has a bug though: it can’t handle an iterator that’s empty.

update_current(cursor, tid)

Update the current object pointers.

tid is the integer tid of the transaction being committed.

upload_blob(cursor, oid, tid, filename)

Upload a blob from a file.

If serial is None, upload to the temporary table.

class RowBatcherStoreTemps(keep_history, binary, batcher_factory=<class 'relstorage.adapters.batch.RowBatcher'>)

Bases: object

A helper class to implement store_temps using a RowBatcher. You must provide an implementation of store_temp_into_batcher() and it must be an upsert. The generic_store_temp_into_batcher() method can be used to help with this.