relstorage.adapters.sqlite.mover

class Sqlite3ObjectMover(database_driver, options, runner=None, version_detector=None, batcher_factory=<class 'relstorage.adapters.sqlite.batch.Sqlite3RowBatcher'>)

Bases: AbstractObjectMover

Parameters:

database_driver – The IDBDriver in use.

on_store_opened(cursor, restart=False)

Create the temporary table for storing objects

replace_temps(cursor, state_oid_tid_iter)

Assumes that store_temps is using an upsert query and simply calls that method.

The same comments apply. In particular, MySQLclient won’t optimize an UPDATE in the same way it does an INSERT.

store_temps(cursor, state_oid_tid_iter)

Uses the cursor’s executemany method to store temporary objects.

Parameters:

state_oid_tid_iter – An iterable over tuples (state, oid_int, tid_int). Data may be None to indicate we should store a NULL.

If there is a more optimal way to implement putting objects in the database, please do so.

• On SQLite, executemany is implemnted in a C looping over the provided iterator. Which it turns out is exactly what the normal execute method also does (it just uses a one-row iterator). So executemany that saves substantial setup overhead dealing with sqlite’s prepared statements.

• On Postgresql, we use COPY for this (unless we’re using the ‘gevent psycopg2’ driver; it’s the only thing that doesn’t support COPY). None of the supported PostgreSQL drivers have a good executemany method, so they should fall back to using our own RowBatcher.

• On Oracle, we use the RowBatcher with a combination of bulk array operations and direct inserts.

• On MySQL, the preferred driver (mysqlclient) has a decent implementation of executemany for INSERT or REPLACE (basically an optimized form of what our RowBatcher does). That implementation is shared with PyMySQL as well, but it must be a simple INSERT statement matching a regular expression. Note that it has a bug though: it can’t handle an iterator that’s empty.