diff --git a/CHANGELOG.md b/CHANGELOG.md index 42c18ee2c..46fe2e1aa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,29 @@ releases, visit GitHub: https://github.com/coleifer/peewee/releases +## 2.5.1 + +This is a relatively small release with a few important bugfixes. + +### Bugs fixed + +* #566, fixed a bug regarding parentheses around compound `SELECT` queries (i.e. `UNION`, `INTERSECT`, etc). +* Fixed unreported bug where table aliases were not generated correctly for compound `SELECT` queries. +* #559, add option to preserve original column order with `pwiz`. Thanks @elgow! +* Fixed unreported bug where selecting all columns from a `ModelAlias` does not use the appropriate `FieldAlias` objects. + +### New features + +* #561, added an option for bulk insert queries to return the list of auto-generated primary keys. See [docs for InsertQuery.return_id_list](http://docs.peewee-orm.com/en/latest/peewee/api.html#InsertQuery.return_id_list). +* #569, added `parse` function to the `playhouse.db_url` module. Thanks @stt! +* Added [hacks](http://docs.peewee-orm.com/en/latest/peewee/hacks.html) section to the docs. Please contribute your hacks! + +### Backwards-incompatible changes + +* Calls to `Node.in_()` and `Node.not_in()` do not take `*args` anymore and instead take a single argument. + +[View commits](https://github.com/coleifer/peewee/compare/2.5.0...2.5.1) + ## 2.5.0 There are a couple new features so I thought I'd bump to 2.5.x. One change Postgres users may be happy to see is the use of `INSERT ... RETURNING` to perform inserts. This should definitely speed up inserts for Postgres, since an extra query is no longer needed to get the new auto-generated primary key. diff --git a/docs/peewee/api.rst b/docs/peewee/api.rst index 1898e024e..1ce9990e9 100644 --- a/docs/peewee/api.rst +++ b/docs/peewee/api.rst @@ -258,6 +258,38 @@ Models found. If more than one row is found, the first row returned by the database cursor will be used. + .. py:classmethod:: get_or_create([defaults=None[, **kwargs]]) + + :param dict defaults: A dictionary of values to set on newly-created model instances. + :param kwargs: Django-style filters specifying which model to get, and what values to apply to new instances. + :returns: A 2-tuple containing the model instance and a boolean indicating whether the instance was created. + + This function attempts to retrieve a model instance based on the provided filters. If no matching model can be found, a new model is created using the parameters specified by the filters and any values in the ``defaults`` dictionary. + + Example **without** ``get_or_create``: + + .. code-block:: python + + # Without `get_or_create`, we might write: + try: + person = Person.get( + (Person.first_name == 'John') & + (Person.last_name == 'Lennon')) + except Person.DoesNotExist: + person = Person.create( + first_name='John', + last_name='Lennon', + birthday=datetime.date(1940, 10, 9)) + + Equivalent code using ``get_or_create``: + + .. code-block:: python + + person, created = Person.get_or_create( + first_name='John', + last_name='Lennon', + defaults={'birthday': datetime.date(1940, 10, 9)}) + .. py:classmethod:: alias() :rtype: :py:class:`ModelAlias` instance diff --git a/docs/peewee/database.rst b/docs/peewee/database.rst index 231921c86..30c1e40a4 100644 --- a/docs/peewee/database.rst +++ b/docs/peewee/database.rst @@ -281,7 +281,7 @@ Example database URLs: * *sqlite:///my_database.db* will create a :py:class:`SqliteDatabase` instance for the file ``my_database.db`` in the current directory. * *postgresql://postgres:my_password@localhost:5432/my_database* will create a :py:class:`PostgresqlDatabase` instance. A username and password are provided, as well as the host and port to connect to. -* *mysql:///my_db* will create a :py:class:`MySQLDatabase` instance for the local MySQL database *my_db*. +* *mysql://user:passwd@ip:port/my_db* will create a :py:class:`MySQLDatabase` instance for the local MySQL database *my_db*. Multi-threaded applications --------------------------- diff --git a/docs/peewee/querying.rst b/docs/peewee/querying.rst index d9beb8f84..2560f555c 100644 --- a/docs/peewee/querying.rst +++ b/docs/peewee/querying.rst @@ -88,12 +88,12 @@ The above approach is slow for a couple of reasons: 3. That's a lot of data (in terms of raw bytes of SQL) you are sending to your database to parse. 4. We are retrieving the *last insert id*, which causes an additional query to be executed in some cases. -You can get a **very significant speedup** by simply wrapping this in a :py:meth:`~Database.transaction`. +You can get a **very significant speedup** by simply wrapping this in a :py:meth:`~Database.atomic`. .. code-block:: python # This is much faster. - with db.transaction(): + with db.atomic(): for data_dict in data_source: Model.create(**data_dict) @@ -102,7 +102,7 @@ The above code still suffers from points 2, 3 and 4. We can get another big boos .. code-block:: python # Fastest. - with db.transaction(): + with db.atomic(): Model.insert_many(data_source).execute() Depending on the number of rows in your data source, you may need to break it up into chunks: @@ -110,7 +110,7 @@ Depending on the number of rows in your data source, you may need to break it up .. code-block:: python # Insert rows 1000 at a time. - with db.transaction(): + with db.atomic(): for idx in range(0, len(data_source), 1000): Model.insert_many(data_source[idx:idx+1000]).execute() @@ -270,20 +270,44 @@ For more information, see the documentation on: Get or create ------------- -While peewee has a :py:meth:`~Model.get_or_create` method, this should really not be used outside of tests as it is vulnerable to a race condition. The proper way to perform a *get or create* with peewee is to rely on the database to enforce a constraint. +While peewee has a :py:meth:`~Model.get_or_create` method, I do not advise you use it. The proper way to perform a *get or create* with peewee is to *create then get*, relying on database constraints to avoid duplicate records. Let's say we wish to implement registering a new user account using the :ref:`example User model `. The *User* model has a *unique* constraint on the username field, so we will rely on the database's integrity guarantees to ensure we don't end up with duplicate usernames: .. code-block:: python try: - with db.transaction(): + with db.atomic(): return User.create(username=username) except peewee.IntegrityError: # `username` is a unique column, so this username already exists, # making it safe to call .get(). return User.get(User.username == username) +The above example first attempts at creation, then falls back to retrieval, relying on the database to enforce a unique constraint. + +If you prefer to attempt to retrieve the record first, you can use :py:meth:`~Model.get_or_create`. This method is implemented along the same lines as the Django function of the same name. You can use the Django-style keyword argument filters to specify your ``WHERE`` conditions. The function returns a 2-tuple containing the instance and a boolean value indicating if the object was created. + +Here is how you might implement user account creation using :py:meth:`~Model.get_or_create`: + +.. code-block:: python + + user, created = User.get_or_create(username=username) + +Suppose we have a different model ``Person`` and would like to get or create a person object. The only conditions we care about when retrieving the ``Person`` are their first and last names, **but** if we end up needing to create a new record, we will also specify their date-of-birth and favorite color: + +.. code-block:: python + + person, created = Person.get_or_create( + first_name=first_name, + last_name=last_name, + defaults={'dob': dob, 'favorite_color': 'green'}) + +Any keyword argument passed to :py:meth:`~Model.get_or_create` will be used in the ``get()`` portion of the logic, except for the ``defaults`` dictionary, which will be used to populate values on newly-created instances. + +For more details check out the documentation for :py:meth:`Model.get_or_create`. + + Selecting multiple records -------------------------- diff --git a/peewee.py b/peewee.py index a8a14f1e2..1d5885b74 100644 --- a/peewee.py +++ b/peewee.py @@ -35,7 +35,7 @@ from functools import wraps from inspect import isclass -__version__ = '2.5.1' +__version__ = '2.6.0' __all__ = [ 'BareField', 'BigIntegerField', @@ -3250,13 +3250,15 @@ class PostgresqlDatabase(Database): register_unicode = True - def _connect(self, database, **kwargs): + def _connect(self, database, encoding=None, **kwargs): if not psycopg2: raise ImproperlyConfigured('psycopg2 must be installed.') conn = psycopg2.connect(database=database, **kwargs) if self.register_unicode: pg_extensions.register_type(pg_extensions.UNICODE, conn) pg_extensions.register_type(pg_extensions.UNICODEARRAY, conn) + if encoding: + conn.set_client_encoding(encoding) return conn def _get_pk_sequence(self, model): @@ -3943,11 +3945,22 @@ def get(cls, *query, **kwargs): @classmethod def get_or_create(cls, **kwargs): + defaults = kwargs.pop('defaults', {}) sq = cls.select().filter(**kwargs) try: - return sq.get() + return sq.get(), False except cls.DoesNotExist: - return cls.create(**kwargs) + try: + params = dict((k, v) for k, v in kwargs.items() + if '__' not in k) + params.update(defaults) + with cls._meta.database.atomic(): + return cls.create(**params), True + except IntegrityError as exc: + try: + return sq.get(), False + except cls.DoesNotExist: + raise exc @classmethod def filter(cls, *dq, **query): diff --git a/playhouse/tests/test_models.py b/playhouse/tests/test_models.py index a176c7229..d3902d540 100644 --- a/playhouse/tests/test_models.py +++ b/playhouse/tests/test_models.py @@ -12,6 +12,21 @@ from playhouse.tests.models import * +in_memory_db = database_initializer.get_in_memory_database() + +class GCModel(Model): + name = CharField(unique=True) + key = CharField() + value = CharField() + number = IntegerField(default=0) + + class Meta: + database = in_memory_db + indexes = ( + (('key', 'value'), True), + ) + + class TestQueryingModels(ModelTestCase): requires = [User, Blog] @@ -289,6 +304,11 @@ def test_model_iter(self): class TestModelAPIs(ModelTestCase): requires = [User, Blog, Category, UserCategory] + def setUp(self): + super(TestModelAPIs, self).setUp() + GCModel.drop_table(True) + GCModel.create_table() + def test_related_name(self): u1 = User.create(username='u1') u2 = User.create(username='u2') @@ -533,11 +553,59 @@ def test_reading(self): self.assertEqual(u2, User.get(User.username == 'u2')) def test_get_or_create(self): - u1 = User.get_or_create(username='u1') - u1_x = User.get_or_create(username='u1') + u1, created = User.get_or_create(username='u1') + self.assertTrue(created) + + u1_x, created = User.get_or_create(username='u1') + self.assertFalse(created) + self.assertEqual(u1.id, u1_x.id) self.assertEqual(User.select().count(), 1) + def test_get_or_create_extended(self): + gc1, created = GCModel.get_or_create( + name='huey', + key='k1', + value='v1', + defaults={'number': 3}) + self.assertTrue(created) + self.assertEqual(gc1.name, 'huey') + self.assertEqual(gc1.key, 'k1') + self.assertEqual(gc1.value, 'v1') + self.assertEqual(gc1.number, 3) + + gc1_db, created = GCModel.get_or_create( + name='huey', + defaults={'key': 'k2', 'value': 'v2'}) + self.assertFalse(created) + self.assertEqual(gc1_db.id, gc1.id) + self.assertEqual(gc1_db.key, 'k1') + + def integrity_error(): + gc2, created = GCModel.get_or_create( + name='huey', + key='kx', + value='vx') + + self.assertRaises(IntegrityError, integrity_error) + + gc2, created = GCModel.get_or_create( + name__ilike='%nugget%', + defaults={ + 'name': 'foo-nugget', + 'key': 'k2', + 'value': 'v2'}) + self.assertTrue(created) + self.assertEqual(gc2.name, 'foo-nugget') + + gc2_db, created = GCModel.get_or_create( + name__ilike='%nugg%', + defaults={'name': 'xx'}) + self.assertFalse(created) + self.assertEqual(gc2_db.id, gc2.id) + + self.assertEqual(GCModel.select().count(), 2) + def test_first(self): users = User.create_users(5) diff --git a/playhouse/tests/test_postgres.py b/playhouse/tests/test_postgres.py index caa2f3827..ee9227453 100644 --- a/playhouse/tests/test_postgres.py +++ b/playhouse/tests/test_postgres.py @@ -396,6 +396,28 @@ def _create_am(self): tags=['alpha', 'beta', 'gamma', 'delta'], ints=[[1, 2], [3, 4], [5, 6]]) + def test_joining_on_array_index(self): + values = [ + ['foo', 'bar'], + ['foo', 'nugget'], + ['baze', 'nugget']] + for tags in values: + ArrayModel.create(tags=tags, ints=[]) + + for value in ['nugget', 'herp', 'foo']: + NormalModel.create(data=value) + + query = (ArrayModel + .select() + .join( + NormalModel, + on=(NormalModel.data == ArrayModel.tags[1])) + .order_by(ArrayModel.id)) + results = [am.tags for am in query] + self.assertEqual(results, [ + ['foo', 'nugget'], + ['baze', 'nugget']]) + def test_array_storage_retrieval(self): am = self._create_am() am_db = ArrayModel.get(ArrayModel.id == am.id) @@ -635,6 +657,31 @@ def test_json_field(self): j_db = self.ModelClass.get(j._pk_expr()) self.assertEqual(j_db.data, data) + def test_joining_on_json_key(self): + JsonModel = self.ModelClass + values = [ + {'foo': 'bar', 'baze': {'nugget': 'alpha'}}, + {'foo': 'bar', 'baze': {'nugget': 'beta'}}, + {'herp': 'derp', 'baze': {'nugget': 'epsilon'}}, + {'herp': 'derp', 'bar': {'nuggie': 'alpha'}}, + ] + for data in values: + JsonModel.create(data=data) + + for value in ['alpha', 'beta', 'gamma', 'delta']: + NormalModel.create(data=value) + + query = (JsonModel + .select() + .join(NormalModel, on=( + NormalModel.data == JsonModel.data['baze']['nugget'])) + .order_by(JsonModel.id)) + results = [jm.data for jm in query] + self.assertEqual(results, [ + {'foo': 'bar', 'baze': {'nugget': 'alpha'}}, + {'foo': 'bar', 'baze': {'nugget': 'beta'}}, + ]) + def test_json_lookup_methods(self): data = { 'gp1': { @@ -756,7 +803,7 @@ def pg93(): @skip_if(lambda: not json_ok()) class TestJsonField(BaseJsonFieldTestCase, ModelTestCase): ModelClass = TestingJson - requires = [TestingJson] + requires = [TestingJson, NormalModel] def jsonb_ok(): if BJson is None: @@ -767,7 +814,7 @@ def jsonb_ok(): @skip_if(lambda: not json_ok()) class TestBinaryJsonField(BaseJsonFieldTestCase, ModelTestCase): ModelClass = BJson - requires = [BJson] + requires = [BJson, NormalModel] def _create_test_data(self): data = [