relstorage.adapters.mover
– IObjectMover implementation¶
IObjectMover implementation.
- class AbstractObjectMover(database_driver, options, runner=None, version_detector=None, batcher_factory=<class 'relstorage.adapters.batch.RowBatcher'>)[source]¶
Bases:
DatabaseHelpersMixin
,ABC
- Parameters:
database_driver – The IDBDriver in use.
- current_object_tids(cursor, oids, timeout=None)[source]¶
Returns the current {oid: tid} for specified object ids.
- get_object_tid_after(cursor, oid, tid)[source]¶
Returns the tid of the next change after an object revision.
Returns None if no later state exists.
- load_before(cursor, oid, tid)[source]¶
Returns the pickle and tid of an object before transaction tid.
Returns (None, None) if no earlier state exists.
- load_current(cursor, oid)[source]¶
Returns the current pickle and integer tid for an object.
oid is an integer. Returns (None, None) if object does not exist.
- load_currents(cursor, oids)[source]¶
Returns the current (oid, state, tid) for specified object ids.
- load_revision(cursor, oid, tid)[source]¶
Returns the pickle for an object on a particular transaction.
Returns None if no such state exists.
- move_from_temp(cursor, tid, txn_has_blobs)[source]¶
Move the temporarily stored objects to permanent storage.
- replace_temps(cursor, state_oid_tid_iter)[source]¶
Assumes that
store_temps
is using an upsert query and simply calls that method.The same comments apply. In particular, MySQLclient won’t optimize an UPDATE in the same way it does an INSERT.
- store_temps(cursor, state_oid_tid_iter)[source]¶
Uses the cursor’s
executemany
method to store temporary objects.- Parameters:
state_oid_tid_iter – An iterable over tuples
(state, oid_int, tid_int)
. Data may be None to indicate we should store a NULL.
If there is a more optimal way to implement putting objects in the database, please do so.
On SQLite,
executemany
is implemnted in a C looping over the provided iterator. Which it turns out is exactly what the normalexecute
method also does (it just uses a one-row iterator). Soexecutemany
that saves substantial setup overhead dealing with sqlite’s prepared statements.On Postgresql, we use COPY for this (unless we’re using the ‘gevent psycopg2’ driver; it’s the only thing that doesn’t support COPY). None of the supported PostgreSQL drivers have a good
executemany
method, so they should fall back to using our own RowBatcher.On Oracle, we use the RowBatcher with a combination of bulk array operations and direct inserts.
On MySQL, the preferred driver (mysqlclient) has a decent implementation of executemany for INSERT or REPLACE (basically an optimized form of what our RowBatcher does). That implementation is shared with PyMySQL as well, but it must be a simple INSERT statement matching a regular expression. Note that it has a bug though: it can’t handle an iterator that’s empty.
- class RowBatcherStoreTemps(keep_history, binary, batcher_factory=<class 'relstorage.adapters.batch.RowBatcher'>)[source]¶
Bases:
object
A helper class to implement
store_temps
using a RowBatcher. You must provide an implementation ofstore_temp_into_batcher()
and it must be an upsert. Thegeneric_store_temp_into_batcher()
method can be used to help with this.